enso-org/enso
1
1
Fork 0
mirror of https://github.com/enso-org/enso.git synced 2024-10-05 17:17:50 +03:00
Code Issues Projects Releases Wiki Activity

Add a markdown style guide (#1022)

Browse Source
This commit is contained in:
Ara Adkins 2020-07-21 13:59:40 +01:00 committed by GitHub
parent 1a38f7c331
commit f5ffbe8fa7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
146 changed files with 2335 additions and 1184 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
.github/ISSUE_TEMPLATE/bug-report.md vendored
View File

@ -1,23 +1,24 @@
---
name: Bug Report
about: Report a bug in Enso
title: ''
labels: 'Type: Bug'
assignees: ''
title: ""
labels: "Type: Bug"
assignees: ""
---
<!--
Please ensure that you are running the latest version of Enso before reporting
Please ensure that you are running the latest version of Enso before reporting
the bug! It may have been fixed since.
-->
### General Summary
<!--
- Please include a high-level description of your bug here.
-->
### Steps to Reproduce
<!--
Please list the reproduction steps for your bug. For example:
@ -37,17 +38,20 @@ Please list the reproduction steps for your bug. For example:
-->
### Expected Result
<!--
- A description of the results you expected from the reproduction steps.
-->
### Actual Result
<!--
- A description of what actually happens when you run these steps.
- Please include any error output if relevant.
-->
### Enso Version
<!--
- Please include the output of `enso --version`.

11
.github/ISSUE_TEMPLATE/epic.md vendored
View File

@ -1,30 +1,33 @@
---
name: Epic
about: Create a new epic for Enso development
title: ''
labels: ''
assignees: ''
title: ""
labels: ""
assignees: ""
---
### Summary
<!--
- This section should summarise the work we want to accomplish during the epic.
-->
### Value
<!--
- A description of the value this epic brings to users.
- The motivation behind this epic.
-->
### Specification
<!--
- The high-level requirements of the epic.
- Any performance requirements for the epic.
-->
### Acceptance Criteria & Test Cases
<!--
- The high-level acceptance criteria for the epic.
- The test plan for the epic.

11
.github/ISSUE_TEMPLATE/feature-request.md vendored
View File

@ -1,23 +1,24 @@
---
name: Feature Request
about: Request a new feature in Enso
title: ''
labels: 'Type: Enhancement'
assignees: ''
title: ""
labels: "Type: Enhancement"
assignees: ""
---
<!--
Please ensure that you check the latest version of Enso to see if your feature
Please ensure that you check the latest version of Enso to see if your feature
has been implemented.
-->
### General Summary
<!--
- Describe the feature you are requesting.
-->
### Motivation
<!--
- A description of the motivation for adding this feature to Enso.
- Ideally this would include use-cases that support the feature.

11
.github/ISSUE_TEMPLATE/task.md vendored
View File

@ -1,30 +1,33 @@
---
name: Task
about: Create a new development task for Enso
title: ''
labels: ''
assignees: ''
title: ""
labels: ""
assignees: ""
---
### Summary
<!--
- A summary of the task.
-->
### Value
<!--
- This section should describe the value of this task.
- This value can be for users, to the team, etc.
-->
### Specification
<!--
- Detailed requirements for the feature.
- The performance requirements for the feature.
-->
### Acceptance Criteria & Test Cases
<!--
- Any criteria that must be satisfied for the task to be accepted.
- The test plan for the feature, related to the acceptance criteria.

15
.github/PULL_REQUEST_TEMPLATE.md vendored
View File

@ -1,18 +1,31 @@
### Pull Request Description
<!--
- Please describe the nature of your PR here, as well as the motivation for it.
- If it fixes an open issue, please mention that issue number here.
-->
### Important Notes
<!--
- Mention important elements of the design.
- Mention any notable changes to APIs.
-->
### Checklist
Please include the following checklist in your PR:
- [ ] The documentation has been updated if necessary.
- [ ] All code conforms to the [Scala](https://github.com/enso-org/enso/blob/main/docs/style-guide/scala.md), [Java](https://github.com/enso-org/enso/blob/main/docs/style-guide/java.md), and [Rust](https://github.com/enso-org/enso/blob/main/docs/style-guide/rust.md) style guides.
- [ ] All code conforms to the
[Scala](https://github.com/enso-org/enso/blob/main/docs/style-guide/scala.md),
[Java](https://github.com/enso-org/enso/blob/main/docs/style-guide/java.md),
and
[Rust](https://github.com/enso-org/enso/blob/main/docs/style-guide/rust.md)
style guides.
- [ ] All documentation and configuration conforms to the
[markdown](https://github.com/enso-org/enso/blob/main/docs/style-guide/markdown.md)
and
[YAML](https://github.com/enso-org/enso/blob/main/docs/style-guide/yaml.md)
style guides.
- [ ] All code has been tested where possible.

53
.github/settings.yml vendored
View File

@ -5,7 +5,9 @@ repository:
name: enso
description: Hybrid visual and textual functional programming.
homepage: https://enso.org
topics: enso, visual, textual, hybrid, functional, language, compiler, interpreter, rts, jit, polyglot
topics:
enso, visual, textual, hybrid, functional, language, compiler, interpreter,
rts, jit, polyglot
private: false
@ -76,7 +78,8 @@ labels:
description: A change that will break a public API or user-facing behaviour
- name: "Change: Non-Breaking"
color: "#ffdce5"
description: A change that will not break a public API or user-facing behaviour
description:
A change that will not break a public API or user-facing behaviour
- name: "Difficulty: Beginner"
color: "#d1e9c4"
@ -208,17 +211,17 @@ branches:
# Require branches to be up to date before merging.
strict: true
contexts:
- "Test Engine (macOS-latest)"
- "Test Engine (ubuntu-latest)"
- "Test Engine (windows-latest)"
- "Build Engine"
- "Rust Check"
- "Rust Lint"
- "Rust Test Native (macOS-latest)"
- "Rust Test Native (ubuntu-latest)"
- "Rust Test Native (windows-latest)"
- "Rust Test WASM"
- "license/cla"
- "Test Engine (macOS-latest)"
- "Test Engine (ubuntu-latest)"
- "Test Engine (windows-latest)"
- "Build Engine"
- "Rust Check"
- "Rust Lint"
- "Rust Test Native (macOS-latest)"
- "Rust Test Native (ubuntu-latest)"
- "Rust Test Native (windows-latest)"
- "Rust Test WASM"
- "license/cla"
enforce_admins: null
restrictions: null
- name: "release/*"
@ -233,17 +236,17 @@ branches:
# Require branches to be up to date before merging.
strict: true
contexts:
- "Test Engine (macOS-latest)"
- "Test Engine (ubuntu-latest)"
- "Test Engine (windows-latest)"
- "Build Engine"
- "Rust Check"
- "Rust Lint"
- "Rust Test Native (macOS-latest)"
- "Rust Test Native (ubuntu-latest)"
- "Rust Test Native (windows-latest)"
- "Rust Test WASM"
- "license/cla"
- "Vulnerability Scan"
- "Test Engine (macOS-latest)"
- "Test Engine (ubuntu-latest)"
- "Test Engine (windows-latest)"
- "Build Engine"
- "Rust Check"
- "Rust Lint"
- "Rust Test Native (macOS-latest)"
- "Rust Test Native (ubuntu-latest)"
- "Rust Test Native (windows-latest)"
- "Rust Test WASM"
- "license/cla"
- "Vulnerability Scan"
enforce_admins: null
restrictions: null

92
.github/workflows/codeql-analysis.yml vendored
View File

@ -22,59 +22,59 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
- name: Checkout repository
uses: actions/checkout@v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: java
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: java
# Set Up Environment
- name: Setup conda
uses: s-weigand/setup-conda@v1
with:
update-conda: false
conda-channels: anaconda, conda-forge
- name: Install FlatBuffers Compiler
run: conda install --freeze-installed flatbuffers=1.12.0
- name: Setup GraalVM Environment
uses: DeLaGuardo/setup-graalvm@2.0
with:
# Set Up Environment
- name: Setup conda
uses: s-weigand/setup-conda@v1
with:
update-conda: false
conda-channels: anaconda, conda-forge
- name: Install FlatBuffers Compiler
run: conda install --freeze-installed flatbuffers=1.12.0
- name: Setup GraalVM Environment
uses: DeLaGuardo/setup-graalvm@2.0
with:
graalvm-version: ${{ env.graalVersion }}.${{ env.javaVersion }}
- name: Set Up SBT
run: |
- name: Set Up SBT
run: |
curl --retry 4 --retry-connrefused -fsSL -o sbt.tgz https://piccolo.link/sbt-${{env.sbtVersion}}.tgz
tar -xzf sbt.tgz
echo ::add-path::$GITHUB_WORKSPACE/sbt/bin/
# Caches
- name: Cache SBT
uses: actions/cache@v2
with:
path: |
~/.sbt
~/.ivy2/cache
~/.cache
key: ${{ runner.os }}-sbt-${{ hashFiles('**build.sbt') }}
restore-keys: ${{ runner.os }}-sbt-
# Caches
- name: Cache SBT
uses: actions/cache@v2
with:
path: |
~/.sbt
~/.ivy2/cache
~/.cache
key: ${{ runner.os }}-sbt-${{ hashFiles('**build.sbt') }}
restore-keys: ${{ runner.os }}-sbt-
# Build
- name: Bootstrap Enso project
run: sbt --no-colors bootstrap
- name: Build Enso
run: sbt --no-colors compile
# Build
- name: Bootstrap Enso project
run: sbt --no-colors bootstrap
- name: Build Enso
run: sbt --no-colors compile
# Analyse the Code
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
# Analyse the Code
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1

53
.github/workflows/docs.yml vendored
View File

@ -5,35 +5,34 @@ on:
branches:
- main
paths:
- 'docs/**'
- "docs/**"
jobs:
checkout:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
repository: 'enso-org/enso-org.github.io'
ref: 'sources'
token: ${{ secrets.ENSO_PAT }}
- name: set identity email
run: git config --global user.email "actions@github.com"
- name: set identity name
run: git config --global user.name "GitHub Actions"
- name: Checkout submodules
run: git submodule update --init
- name: Update submodules
id: status
run: |
git submodule update --init
git submodule update --remote --merge
if [ -n "$(git status --porcelain)" ]; then
echo "::set-output name=has_changes::1"
fi
- name: Push changes
run: |
git add .
git commit -m "Update submodules"
git push origin sources
if: steps.status.outputs.has_changes == '1'
- uses: actions/checkout@v2
with:
repository: "enso-org/enso-org.github.io"
ref: "sources"
token: ${{ secrets.ENSO_PAT }}
- name: set identity email
run: git config --global user.email "actions@github.com"
- name: set identity name
run: git config --global user.name "GitHub Actions"
- name: Checkout submodules
run: git submodule update --init
- name: Update submodules
id: status
run: |
git submodule update --init
git submodule update --remote --merge
if [ -n "$(git status --porcelain)" ]; then
echo "::set-output name=has_changes::1"
fi
- name: Push changes
run: |
git add .
git commit -m "Update submodules"
git push origin sources
if: steps.status.outputs.has_changes == '1'

11
.github/workflows/rust.yml vendored
View File

@ -2,9 +2,9 @@ name: Rust CI
on:
push:
branches: [ main, "release/*" ]
branches: [main, "release/*"]
pull_request:
branches: [ "*" ]
branches: ["*"]
env:
wasmpackVersion: 0.8.1
@ -154,8 +154,11 @@ jobs:
# We could use cargo install wasm-pack, but that takes 3.5 minutes
# compared to a few seconds.
env:
WASMPACKURL: https://github.com/rustwasm/wasm-pack/releases/download/v${{ env.wasmpackVersion }}
WASMPACKDIR: wasm-pack-v${{ env.wasmpackVersion }}-x86_64-unknown-linux-musl
WASMPACKURL:
https://github.com/rustwasm/wasm-pack/releases/download/v${{
env.wasmpackVersion }}
WASMPACKDIR:
wasm-pack-v${{ env.wasmpackVersion }}-x86_64-unknown-linux-musl
shell: bash
run: |
curl -L "$WASMPACKURL/$WASMPACKDIR.tar.gz" | tar -xz -C .

9
.github/workflows/scala.yml vendored
View File

@ -2,9 +2,9 @@ name: Engine CI
on:
push:
branches: [ main, "release/*" ]
branches: [main, "release/*"]
pull_request:
branches: [ "*" ]
branches: ["*"]
env:
# Please ensure that this is in sync with graalVersion in build.sbt
@ -99,7 +99,10 @@ jobs:
run: ./runner.jar --run tools/ci/Test.enso
- name: Test the Repl in the Uberjar
shell: bash
run: cat tools/ci/artifact-test/repl/ReplTest.in | ./runner.jar --repl | diff -q tools/ci/artifact-test/repl/ReplTest.out - && echo "Test successful"
run:
cat tools/ci/artifact-test/repl/ReplTest.in | ./runner.jar --repl |
diff -q tools/ci/artifact-test/repl/ReplTest.out - && echo "Test
successful"
# This job is responsible for building the artifacts
build:

7
.gitignore vendored
View File

@ -31,6 +31,13 @@ dist
cabal-dev
.stack-work/
############
## NodeJS ##
############
node_modules/
package-lock.json
############
## System ##
############

3
.prettierignore Normal file
View File

@ -0,0 +1,3 @@
# Build Artefacts
target/

4
.prettierrc.json Normal file
View File

@ -0,0 +1,4 @@
{
"printWidth": 80,
"proseWrap": "always"
}

12
README.md
View File

@ -22,6 +22,7 @@
</p>
### Fluidly Combining Worlds
<p>
<a href="https://github.com/enso-org/enso/actions">
<img src="https://github.com/enso-org/enso/workflows/Engine%20CI/badge.svg"
@ -59,6 +60,7 @@ compiler, type-checker, runtime and language server. These components implement
Enso the language in its entirety, and are usable in isolation.
### Getting Started
While we currently don't have any official releases of Enso, nightly builds can
be obtained from the
[CI artifacts](https://github.com/enso-org/enso/actions?query=workflow%3A%22Engine+CI%22).
@ -70,12 +72,14 @@ developer documentation rendered [here](https://dev.enso.org), or in the
repository [here](./docs).
### Building
The project builds on any platform where [GraalVM](https://graalvm.org) can run.
You will need the source code, and [`sbt`](https://www.scala-sbt.org/). For more
information, please read the detailed instructions in
[CONTRIBUTING.md](./docs/CONTRIBUTING.md).
### Enso's Design
If you would like to gain a better understanding of the principles on which Enso
is based, or just delve into the why's and what's of Enso's design, please take
a look in the [`docs/` folder](./docs/). It is split up into subfolders for each
@ -91,6 +95,7 @@ project understand the reasoning behind the code, but also to act as a record of
the decisions that have been made through Enso's evolution.
### License
This repository is licensed under the
[Apache 2.0](https://opensource.org/licenses/apache-2.0), as specified in the
[LICENSE](https://github.com/enso-org/enso/blob/main/LICENSE) file.
@ -101,6 +106,7 @@ also allowing us to release commercial products on top of the platform,
including Enso Cloud and Enso Enterprise server managers.
### Contributing to Enso
Enso is a community-driven open source project which is and will always be open
and free to use. We are committed to a fully transparent development process and
highly appreciate every contribution. If you love the vision behind Enso and you
@ -113,9 +119,9 @@ our [chat](http://chat.luna-lang.org/), and take a look at our
describes all the ways in which you can help out with the project, as well as
provides detailed instructions for building and hacking on Enso.
If you believe that you have found a security vulnerability in Enso, or that
you have a bug report that poses a security risk to Enso's users, please take
a look at our [security guidelines](./docs/SECURITY.md) for a course of action.
If you believe that you have found a security vulnerability in Enso, or that you
have a bug report that poses a security risk to Enso's users, please take a look
at our [security guidelines](./docs/SECURITY.md) for a course of action.
<a href="https://github.com/enso-org/enso/graphs/contributors">
<img src="https://opencollective.com/enso-language/contributors.svg?width=890&button=false">

5
distribution/launcher/README.md
View File

@ -6,6 +6,7 @@
> distribution. It should be completed when the launcher is implemented.
## Installation
The Enso distribution is portable, you can put this directory wherever you want
on your system. For example you can put it at `~/.enso`.
@ -13,12 +14,14 @@ It is a good idea to add the `bin` directory to your system `PATH` so that you
can run Enso just by typing `enso` in any directory.
## Usage
> Actionables for this section:
>
> - Fill in examples showing how to use the launcher.
## Distribution structure
> Actionables for this section:
>
> - This should explain the structure of enso distribution directory, based on
> the [documentation](../../docs/distribution/distribution.md).
> the [documentation](../../docs/distribution/distribution.md).

6
distribution/std-lib/Base/package.yaml
View File

@ -1,5 +1,5 @@
license: APLv2
name: Base
version: '0.0.1'
author: 'Enso Team <contact@enso.org>'
maintainer: 'Enso Team <contact@enso.org>'
version: "0.0.1"
author: "Enso Team <contact@enso.org>"
maintainer: "Enso Team <contact@enso.org>"

16
docs/CODE_OF_CONDUCT.md
View File

@ -7,6 +7,7 @@ order: 3
---
# The Enso Code of Conduct
This document contains the code of conduct for _all_ venues associated with the
Enso project.
@ -45,13 +46,14 @@ Enso project.
- Private harassment is also unacceptable. No matter who you are, if you feel
you have been or are being harassed or made uncomfortable by a community
member, please contact one of the [moderators](#moderators) immediately, or
send an email to the [moderator team](#mod_team).
Whether you're a regular contributor or a newcomer, we care about making this
community a safe place for you and we've got your back.
send an email to the [moderator team](#mod_team). Whether you're a regular
contributor or a newcomer, we care about making this community a safe place
for you and we've got your back.
- Likewise any spamming, trolling, flaming, baiting or other attention-stealing
behaviour is not welcome.
## Moderation
These are the policies for upholding our community's standards of conduct. If
you feel that a thread needs moderation, please contact the
[Enso moderation team](#moderators).
@ -94,10 +96,13 @@ intent and forgive as long as there is an atmosphere of trust.
The enforcement policies listed above apply to all official Enso venues. This
includes the official discord and GitHub repositories under `enso-org`.
> Adapted from the [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling)
> as well as the [Contributor Covenant v1.3.0](https://www.contributor-covenant.org/version/1/3/0/).
> Adapted from the
> [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling)
> as well as the
> [Contributor Covenant v1.3.0](https://www.contributor-covenant.org/version/1/3/0/).
### Moderators
The following members of the Enso organisation can be contacted as moderators:
- [`@joenash`](https://github.com/joenash)
@ -107,6 +112,7 @@ Alternatively, you can send an email to the moderation team at
[moderators@enso.org](mailto:moderators@enso.org).
## Credit
This code of conduct is adapted from the [Rust](https://rust-lang.org) code of
conduct. Many thanks to the Rust community for being such an exemplar of the
open-source spirit!

121
docs/CONTRIBUTING.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Contributing to Enso
Thank you for your interest in contributing to Enso! We believe that only
through community involvement can Enso be the best it can be! There are a whole
host of ways to contribute, and every single one is appreciated. The major
@ -22,8 +23,9 @@ sections of this document are linked below:
- [Design Documentation](#design-documentation)
- [System Requirements](#system-requirements)
- [Getting the Sources](#getting-the-sources)
- [Getting Set Up (Rust)](#getting-set-up-rust)
- [Getting Set Up (JVM)](#getting-set-up-jvm)
- [Getting Set Up \(Rust\)](#getting-set-up-rust)
- [Getting Set Up \(JVM\)](#getting-set-up-jvm)
- [Getting Set Up \(Documentation\)](#getting-set-up-documentation)
- [Building Enso](#building-enso)
- [Running Enso](#running-enso)
- [Pull Requests](#pull-requests)
@ -32,13 +34,13 @@ sections of this document are linked below:
- [Out-of-Tree Contributions](#out-of-tree-contributions)
- [Helpful Documentation and Links](#helpful-documentation-and-links)
<!-- /MarkdownTOC -->
All contributions to Enso should be in keeping with our
[Code of Conduct](./CODE_OF_CONDUCT.md).
## The Contributor License Agreement
As part of your first contribution to this repository, you need to accept the
Contributor License Agreement. You will automatically be asked to sign the CLA
when you make your first pull request.
@ -50,6 +52,7 @@ The CLA you sign applies to all repositories associated with the Enso project,
so you will only have to sign it once at the start of your contributions.
## Issues
If you're wanting to get involved with Enso's development and are looking for
somewhere to start, you can check out the following tags in our issues:
@ -60,20 +63,22 @@ You can use the "Size" and "Difficulty" labels that should be assigned to every
issue to get a better idea of how much work a given issue might be.
## Feature Enhancements
If you feel like you have a suggestion for a change to the way that Enso works
as a language, please take a look at the
[Enso RFC process](./rfcs/README.md) to learn how to file an RFC for the
project.
as a language, please take a look at the [Enso RFC process](./rfcs/README.md) to
learn how to file an RFC for the project.
In essence, the RFC process provides a way to propose major changes to the
language, the compiler, and the runtime in a way that ensures that they get
seen and discussed by all the major stakeholders involved.
language, the compiler, and the runtime in a way that ensures that they get seen
and discussed by all the major stakeholders involved.
If, on the other hand, you're asking for a smaller feature, please feel free to
submit a [feature request](https://github.com/enso-org/enso/issues/new?assignees=&labels=Type%3A+Enhancement&template=feature-request.md&title=)
submit a
[feature request](https://github.com/enso-org/enso/issues/new?assignees=&labels=Type%3A+Enhancement&template=feature-request.md&title=)
to the repository.
## Bug Reports
While it's never great to find a bug, they are a reality of software and
software development! We can't fix or improve on the things that we don't know
about, so report as many bugs as you can! If you're not sure whether something
@ -84,10 +89,11 @@ users of Enso, please look at our [security guidelines](./SECURITY.md).**
Even though GitHub search can be a bit hard to use sometimes, we'd appreciate if
you could
[search](https://github.com/enso-org/enso/search?q=&type=Issues&utf8=%E2%9C%93) for
your issue before filing a bug as it's possible that someone else has already
reported the issue. We know the search isn't the best, and it can be hard to
know what to search for, so we really don't mind if you do submit a duplicate!
[search](https://github.com/enso-org/enso/search?q=&type=Issues&utf8=%E2%9C%93)
for your issue before filing a bug as it's possible that someone else has
already reported the issue. We know the search isn't the best, and it can be
hard to know what to search for, so we really don't mind if you do submit a
duplicate!
Opening an issue is as easy as following
[this link](https://github.com/enso-org/enso/issues/new?template=bug-report.md)
@ -101,11 +107,13 @@ output of `enso --version`, as that will let us know if the bug is Operating
System or Architecture specific.
## Hacking on Enso
This will get you up and running for Enso development, with only a minimal
amount of setup required. Enso's build system is fairly simple, allowing you to
bootstrap the compiler as long as you have a minimal set of tools.
### Design Documentation
If you're going to start contributing to Enso, it is often a good idea to take a
look at the design documentation for the language. These files explain provide
both a rigorous specification of Enso's design, but also insight into the _why_
@ -115,6 +123,7 @@ These can be found in [`docs/`](README.md), and are organised by the part of the
compiler that they relate to.
### System Requirements
In order to build and run Enso you will need the following tools:
- [sbt](https://www.scala-sbt.org/) with the same version as specified in
@ -142,6 +151,7 @@ The flatbuffers `flatc` compiler can be installed from the following locations:
- MacOS users can install it via homebrew (`brew install flatbuffers`).
### Getting the Sources
Given you've probably been reading this document on GitHub, you might have an
inkling where to look!. You can clone Enso using two methods:
@ -161,6 +171,7 @@ git clone git@github.com:enso-org/enso.git
```
### Getting Set Up (Rust)
This project currently requires a specific nightly rust toolchain, as well as a
special set-up step in SBT. To get this project set up, you can run the
following commands:
@ -180,8 +191,9 @@ Please note that once the parser is integrated into the SBT build, the
rust-related commands will be automatically performed for you.
### Getting Set Up (JVM)
In order to properly build the `runtime` component, the JVM running SBT needs
to have some dependency JARs available in its module path at startup. To ensure
In order to properly build the `runtime` component, the JVM running SBT needs to
have some dependency JARs available in its module path at startup. To ensure
they are available, before running any compilation or other tasks, these
dependencies should be prepared. To do so, run the following command in the
repository root directory:
@ -196,7 +208,32 @@ because SBT has to be launched again anyway to pick up these JARs at startup.
Bootstrap has to be run only when building the project for the first time
**and** after each change of Graal version.
### Getting Set Up (Documentation)
We enforce automated formatting of all of our documentation and configuration
using the fairly common [prettier](https://prettier.io) automatic formatter. You
can install prettier for our project by running the following command:
```bash
npm install
```
This does, however, mean that you have to have node installed on your system.
Please follow the guidelines [above](#getting-set-up-rust) to install node if
you have not already done so.
The version if prettier is forced by our
[`package-lock.json`](../package-lock.json) in order for us to make formatting
bumps all at once.
You can format all of our documentation and configuration as follows:
```bash
npx prettier --write <dir>
```
### Building Enso
There are multiple projects in this repository, but all can be built, run and
tested using `sbt`. As long as your configuration is correct, with the correct
versions of SBT and GraalVM, the same steps can be followed on all of our
@ -211,6 +248,7 @@ to provide work-arounds for building on that platform while it is still in an
unstable state.
#### Building Enso Components
In order to build a specific component (e.g. `runtime`), please follow the
following steps.
@ -226,6 +264,7 @@ shell will execute the appropriate thing. Furthermore we have `testOnly` and
or benchmarks to run (e.g. `testOnly *FunctionArguments*`).
#### Building the Interpreter CLI Fat Jar
In order to build a fat jar with the CLI component, run the `assembly` task
inside the `runner` subproject:
@ -242,6 +281,7 @@ JAVA_HOME=<PATH_TO_GRAAL_HOME> ./runner.jar <CLI_ARGS>
```
#### Building the Launcher Native Binary
If you want to build the native launcher binary, you need to ensure that the
Native Image component is installed in your GraalVM distribution. To install it,
run:
@ -257,17 +297,19 @@ sbt launcher/buildNativeImage
```
#### Testing Enso
Running the tests for the JVM enso components is as simple as running
`sbt / test`. To test the Rust components you can run `cargo test`. Finally, you
can run the WASM tests for the rust components by using `./run --test-wasm`.
#### Installing the Jupyter kernel
Enso has a highly experimental and not-actively-maintained Jupyer Kernel.
To run it:
Enso has a highly experimental and not-actively-maintained Jupyer Kernel. To run
it:
1. Build (or download from the CI server) the CLI Fat Jar.
2. Fill in the `engine/language-server/jupyter-kernel/enso/kernel.json`
file, providing correct paths to the `runner.jar` distribution and GraalVM
2. Fill in the `engine/language-server/jupyter-kernel/enso/kernel.json` file,
providing correct paths to the `runner.jar` distribution and GraalVM
`JAVA_HOME`.
3. Run:
@ -278,6 +320,7 @@ jupyter kernelspec install <ROOT_OF_THIS_REPO>/engine/language-server/jupyter-ke
Congratulations, your Jupyter Kernel should now be installed and ready to use.
#### Passing Debug Options
GraalVM provides some useful debugging options, including the ability to output
the compilation graph during JIT optimisation, and the ASM generated by the JIT.
@ -304,6 +347,7 @@ withDebug benchOnly --showCompilations -- RecursionBenchmark
```
#### Working with Assembly
In order to examine the assembly generated by GraalVM and HotSpot you need to
provide your JVM install with a dynamic library that supports the dumping of
assembly. It can be acquired for MacOS and Linux
@ -314,16 +358,16 @@ please choose one best suited for you.
Once you have a copy of the dynamic library, it needs to be placed in
`$JVM_HOME/lib/server`.
#### Native Image
Native image is a capability provided alongside GraalVM that allows the
generation of native executables from JVM language programs (such as the Enso
interpreter itself). However, it results in significantly degraded peak
performance, so it is not part of our roadmap currently.
If you would like to experiment with it, you can execute the `buildNativeImage`
command in the sbt shell while inside the `runner` project. Please note
that while the command is available at the moment, and you are welcome to
command in the sbt shell while inside the `runner` project. Please note that
while the command is available at the moment, and you are welcome to
[report an issue](https://github.com/enso-org/enso/issues/new?assignees=&labels=Type%3A+Bug&template=bug-report.md&title=)
with the functionality, any bugs you report will _not_ be considered high
priority.
@ -331,6 +375,7 @@ priority.
**WE CURRENTLY DO NOT SUPPORT THE NATIVE IMAGE BUILD.**
#### Using IntelliJ
Internally, most of the developers working on the Enso project use IntelliJ as
their primary IDE. To that end, what follows is a basic set of instructions for
getting the project into a working state in IntelliJ.
@ -364,8 +409,8 @@ make use of the IntelliJ debugger. In order to get debugging working, you will
need to follow these steps:
1. Go to Run -> Edit Configurations.
2. Click the `+` button in the header of the 'Run/Debug Configurations'
dialogue that pops up.
2. Click the `+` button in the header of the 'Run/Debug Configurations' dialogue
that pops up.
3. Select 'Remote' and name the new configuration appropriately.
4. In the options for that configuration select 'Listen to remote JVM' under
'Debugger mode:'
@ -383,6 +428,7 @@ need to follow these steps:
configuration as they are specific to your machine.
#### Troubleshooting
If you are having issues building Enso, please check the list below before
filing an issue with us.
@ -402,13 +448,14 @@ If your problem was not listed above, please
in our issue tracker and we will get back to you as soon as possible.
### Running Enso
The only component in this repository with a proper executable is the Enso
interpreter. It can be run using the sbt `run` command in the project
`runner` and provides a rudimentary command-line interface to the basic
capabilities of the interpreter.
Detailed information on the flags it supports can be obtained by executing `run
--help`, but the primary functionality is as follows:
The only component in this repository with a proper executable is the Enso
interpreter. It can be run using the sbt `run` command in the project `runner`
and provides a rudimentary command-line interface to the basic capabilities of
the interpreter.
Detailed information on the flags it supports can be obtained by executing
`run --help`, but the primary functionality is as follows:
- `--new PATH`: Creates a new Enso project at the location spcified by `PATH`.
- `--run PATH`: Executes the interpreter on the Enso source specified by `PATH`.
@ -416,12 +463,14 @@ Detailed information on the flags it supports can be obtained by executing `run
project.
#### Language Server Mode
The Language Server can be run using the `--server` option. It requires also a
content root to be provided (`--root-id` and `--path` options). Command-line
interface of the runner prints all server options when you execute it with
`--help` option.
Below are options uses by the Language Server:
- `--server`: Runs the Language Server
- `--root-id <uuid>`: Content root id.
- `--path <path>`: Path to the content root.
@ -446,6 +495,7 @@ specify the following options:
- `--port`: The port on `interface` where the socket will be opened (e.g. `80`).
## Pull Requests
Pull Requests are the primary method for making changes to Enso. GitHub has
[fantastic documentation](https://help.github.com/articles/about-pull-requests/)
on using the pull request feature. Enso uses the 'fork-and-pull' model of
@ -472,7 +522,8 @@ every push to make sure that you can't forget.
please feel free to ask for a specific person if you've worked with them in a
specific area before!
- If you have questions, or would like to begin the review process before your
PR is 'done', please use the [Draft Pull Requests](https://github.blog/2019-02-14-introducing-draft-pull-requests/)
PR is 'done', please use the
[Draft Pull Requests](https://github.blog/2019-02-14-introducing-draft-pull-requests/)
feature on GitHub. Doing so will allow you to make use of our CI
infrastructure as part of your development process.
@ -482,6 +533,7 @@ please feel free to discuss the suggestions and comments! We can only achieve
the best results through open collaboration.
## Documentation
Documentation improvements are very welcome! For now, the main documentation
available is the _developer_ documentation for the language, which can be found
at the [dev docs site](https://dev.enso.org). The source for this documentation
@ -495,6 +547,7 @@ To find documentation-related issues, sort by the
label.
## Issue Triage
Sometimes issues can be left open long after the bug has been fixed. Other
times, a bug might go stale because something has changed in the meantime.
@ -512,9 +565,11 @@ If you're looking for somewhere to start, take a look at the
issue label, as well as the
[Status: Help Wanted](https://github.com/enso-org/enso/labels/Status%3A%20Help%20Wanted)
and
[Status: Good First Issue](https://github.com/enso-org/enso/labels/Status%3A%20Good%20First%20Issue) labels.
[Status: Good First Issue](https://github.com/enso-org/enso/labels/Status%3A%20Good%20First%20Issue)
labels.
## Out-of-Tree Contributions
As helpful as contributing to Enso directly is, it can also be just as helpful
to contribute in other ways outside this repository:
@ -522,9 +577,9 @@ to contribute in other ways outside this repository:
[StackOverflow](https://stackoverflow.com/questions/tagged/enso).
## Helpful Documentation and Links
For people new to Enso, and just starting to contribute, or even for more
seasoned developers, some useful places to look for information are:
- The [design documentation](./README.md).
- The community! Don't be afraid to ask questions.

15
docs/README.md
View File

@ -7,15 +7,16 @@ order: 0
---
# Enso Engine Developer Documentation
This folder contains the documentation for the implementation of the Enso
programming language. The documentation is broken up by subject, within the
below-listed categories, and each subject combines information about the
design, specification, and implementation of the feature to which it pertains.
below-listed categories, and each subject combines information about the design,
specification, and implementation of the feature to which it pertains.
We provide a number of useful resources for getting a quick understanding of
the Enso project:
We provide a number of useful resources for getting a quick understanding of the
Enso project:
- [**The Enso Philosophy:**](./enso-philosophy.md) Information on the design
- [**The Enso Philosophy:**](./enso-philosophy.md) Information on the design
philosophy behind Enso, and why we build things in the way we do.
- [**Contributing Guidelines:**](./CONTRIBUTING.md) Information for people
wanting to contribute to Enso (in many different ways).
@ -23,8 +24,8 @@ the Enso project:
members of our community, developers and users alike.
- [**Security Guidelines:**](./SECURITY.md) Security guidelines for the Enso
project, including supported versions and our vulnerability reporting process.
- [**Getting Enso:**](./getting-enso.md) Instructions for how to get and run
a copy of Enso.
- [**Getting Enso:**](./getting-enso.md) Instructions for how to get and run a
copy of Enso.
It is broken up into categories as follows:

5
docs/SECURITY.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Security Policy
This document outlines the security policy for Enso and its libraries.
> **If you believe that you have found a vulnerability in Enso or one of its
@ -21,12 +22,13 @@ This document outlines the security policy for Enso and its libraries.
<!-- /MarkdownTOC -->
## Supported Versions
Security updates for Enso are provided for the versions shown below with a
:white_check_mark: next to them. No other versions have security updates
provided.
| Version | Supported |
|-------------|--------------------|
| ----------- | ------------------ |
| `main@HEAD` | :white_check_mark: |
| `wip/*` | :x: |
@ -34,6 +36,7 @@ Please see our [release policy](./distribution/release-policy.md) for more
information on how we support released versions.
## Reporting a Vulnerability
If you believe that you've found a security vulnerability in the Enso codebase
or one of the libraries maintained in this repository, please contact
[security@enso.org](mailto:security@enso.org) and provide details of the bug.

1
docs/debugger/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Debugger
The Enso Debugger allows amongst other things, to execute arbitrary expressions
in a given execution context - this is used to implement a debugging REPL. The
REPL can be launched when triggering a breakpoint in the code.

30
docs/debugger/protocol.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Enso Debugger Protocol Message Specification
Binary Protocol for the Debugger is used in communication between the runtime
and tools exploiting the REPL/debugger functionalities. It can be used to
implement a simple REPL or add debugging capabilities to the editor.
@ -32,6 +33,7 @@ There are some helper types used within the debugger's protocol. These are
specified here.
### `ObjectRepr`
External representation of arbitrary values returned by the REPL (internally
these are of type `Object`).
@ -44,6 +46,7 @@ interface ObjectRepr {
```
### `StackTraceElement`
Represents a line of the stack trace. Corresponds to
`java.lang.StackTraceElement`.
@ -57,6 +60,7 @@ interface StackTraceElement {
```
### `Exception`
Represents an exception that may have been raised during requested execution.
```typescript
@ -68,20 +72,21 @@ interface Exception {
```
### `Binding`
Represents a single binding in the current scope.
```typescript
interface Binding {
name: String;
value: ObjectRepr;
name: String;
value: ObjectRepr;
}
```
## Messages
All endpoints accept messages of type `Request` and return a `Response`.
These messages contain unions that contain the actual payload specified for each
type of message.
All endpoints accept messages of type `Request` and return a `Response`. These
messages contain unions that contain the actual payload specified for each type
of message.
```idl
namespace org.enso.polyglot.debugger.protocol;
@ -118,6 +123,7 @@ notification. This means that this function should not return before sending the
session exit request.
#### Notification
```idl
namespace org.enso.polyglot.protocol.debugger;
@ -125,13 +131,15 @@ table SessionStartNotification {}
```
### Evaluation
Evaluates an arbitrary expression in the current execution context.
Responds with either a message with the value of successfully evaluated
Responds with either a message with the value of successfully evaluated
expression or a message with an exception that has been raised during
evaluation.
#### Request
```idl
namespace org.enso.polyglot.protocol.debugger;
@ -141,6 +149,7 @@ table ReplEvaluationRequest {
```
#### Response
```idl
namespace org.enso.polyglot.protocol.debugger;
@ -154,10 +163,12 @@ table ReplEvaluationFailure {
```
### List Bindings
Lists all the bindings available in the current execution scope and sends them
Lists all the bindings available in the current execution scope and sends them
back.
#### Request
```idl
namespace org.enso.polyglot.protocol.debugger;
@ -165,6 +176,7 @@ table ReplListBindingsRequest {}
```
#### Response
```idl
namespace org.enso.polyglot.protocol.debugger;
@ -174,6 +186,7 @@ table ReplListBindingsResult {
```
### Session Exit
Terminates this REPL session (and resumes normal program execution).
The last result of Evaluation will be returned from the instrumented node or if
@ -182,9 +195,10 @@ no expressions have been evaluated, unit is returned.
This request must always be sent at the end of REPL session, as otherwise the
program will never resume. It does not return any response. It is important to
note that a thread calling `sendBinary` with this message will never return, as
control will be passed to the interpreter.
control will be passed to the interpreter.
#### Request
```idl
namespace org.enso.polyglot.protocol.debugger;

3
docs/distribution/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso Distribution
Documents in this section deal with the process of packaging both Enso and its
dependencies, and Enso projects for use by our users.
@ -21,4 +22,4 @@ dependencies, and Enso projects for use by our users.
- [**Launcher CLI:**](./launcher-cli.md) Explanation of the command-line
interface of the launcher.
- [**Licences:**](licences.md) Information on gathering license information of
dependencies included in the distribution.
dependencies included in the distribution.

46
docs/distribution/distribution.md
View File

@ -7,8 +7,9 @@ order: 1
---
# The Enso Distribution
This document provides a specification of how the Enso distribution should
be structured and how it should behave.
This document provides a specification of how the Enso distribution should be
structured and how it should behave.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -24,10 +25,11 @@ be structured and how it should behave.
<!-- /MarkdownTOC -->
## Universal Launcher
The [universal launcher](./launcher.md) should be able to launch the
proper version of Enso executable based on the version specified in the project
being run, or use the default version if none specified. It should also be able
to launch other Enso components, provided as
The [universal launcher](./launcher.md) should be able to launch the proper
version of Enso executable based on the version specified in the project being
run, or use the default version if none specified. It should also be able to
launch other Enso components, provided as
[plugins](./launcher.md#running-plugins).
> This launcher is under development. Until it is in a ready-to-use state, the
@ -38,11 +40,13 @@ to launch other Enso components, provided as
> it must have full control over which JVM is chosen and its parameters.
## Enso Distribution Layout
Enso is distributed as a portable package that can be extracted anywhere on the
system and run. It can also be installed for the local user into system-defined
directories, as explained below.
### Portable Enso Distribution Layout
All files in the directory structure, except for configuration, can be safely
removed and the launcher will re-download them if needed.
@ -80,6 +84,7 @@ extraction-location
```
### Installed Enso Distribution Layout
After installation, the directory structure is following:
```
@ -120,12 +125,13 @@ If not set, each of these three environment variables defaults to the following
value, depending on the system:
| | Linux | macOS | Windows |
|-------------------------|--------------------------------------------------------------------|-----------------------------------------------|------------------------------|
| ----------------------- | ------------------------------------------------------------------ | --------------------------------------------- | ---------------------------- |
| `ENSO_DATA_DIRECTORY` | `$XDG_DATA_HOME/enso/` which defaults to `$HOME/.local/share/enso` | `$HOME/Library/Application Support/org.enso/` | `%LocalAppData%/enso` |
| `ENSO_CONFIG_DIRECTORY` | `$XDG_CONFIG_HOME/enso/` which defaults to `$HOME/.config/enso` | `$HOME/Library/Preferences/org.enso/` | `%LocalAppData%/enso/config` |
| `ENSO_BIN_DIRECTORY` | `$XDG_BIN_HOME` which defaults to `$HOME/.local/bin` | `$HOME/.local/bin` | `%LocalAppData%/enso/bin` |
### Installing from a Portable Distribution
After downloading and extracting the portable distribution, the user can run
`extraction-location/bin/enso install distribution` to install it locally. This
will copy the files from the portable distribution into the installed locations
@ -140,6 +146,7 @@ The installed distribution can be removed by running
`enso uninstall distribution`.
## Layout of an Enso Version Package
This section describes the structure of a single version distribution. This
system is intended to be implemented first and used e.g. for the Enso nightly
builds / releases.
@ -170,19 +177,21 @@ enso-1.0.0
└── Text.enso
```
> **Implementation Note:**
> This structure makes use of deep nesting, which may give some with knowledge
> of Windows' path-name limits pause (windows paths are historically limited to
> 256 characters). However, there is no special action required to handle this
> limit as long as we are building on top of the JVM. The JVM automatically
> inserts the `\\?\` prefix required to bypass the windows path length limit.
> **Implementation Note:** This structure makes use of deep nesting, which may
> give some with knowledge of Windows' path-name limits pause (windows paths are
> historically limited to 256 characters). However, there is no special action
> required to handle this limit as long as we are building on top of the JVM.
> The JVM automatically inserts the `\\?\` prefix required to bypass the windows
> path length limit.
### Standard Library
The standard library is a set of libraries shipped with the compiler.
Whether a given package belongs to standard library can be a bit of an
arbitrary choice, but the following are some guidelines:
1. Fundamental packages – basic collections and utilities should be a part
of standard library.
The standard library is a set of libraries shipped with the compiler. Whether a
given package belongs to standard library can be a bit of an arbitrary choice,
but the following are some guidelines:
1. Fundamental packages – basic collections and utilities should be a part of
standard library.
2. Packages relying on the compiler internals (e.g. the internal object
representation). An example of such a package would be `Generic`, exposing
reflective access to Enso objects.
@ -190,6 +199,7 @@ arbitrary choice, but the following are some guidelines:
traces etc.
### Resolvers
**Note** This system is not implemented yet.
A resolver is a manifest containing library versions that are automatically

51
docs/distribution/launcher-cli.md
View File

@ -7,6 +7,7 @@ order: 5
---
# Enso Launcher CLI
This document describes available command-line options of the Enso launcher.
> The actionables for this document are:
@ -42,11 +43,12 @@ This document describes available command-line options of the Enso launcher.
## Commands
### `new`
Create a new, empty project in a specified directory.
By default uses the `default` enso version, which can be overriden with
`--version`.
Create a new, empty project in a specified directory. By default uses the
`default` enso version, which can be overriden with `--version`.
Examples:
```bash
> enso new project1 --path /somewhere/on/the/filesystem
# creates project called project1 in the specified directory
@ -56,29 +58,35 @@ Examples:
```
### `install engine`
Installs a specific version of the Enso engine.
Examples:
```bash
> enso install engine 2.0.1
```
### `uninstall engine`
Uninstalls a specific version of the Enso engine.
Examples:
```bash
> enso uninstall engine 2.0.1
```
### `install distribution`
Installs a portable Enso distribution into system-defined directories, as
explained in
[Installed Enso Distribution Layout](./distribution.md#installed-enso-distribution-layout).
By default, it asks the user for confirmation, but this can be skipped by
adding a `--yes` flag.
By default, it asks the user for confirmation, but this can be skipped by adding
a `--yes` flag.
Examples:
```
> extraction-location/bin/enso install distribution
This will install Enso to ~/.local/share/enso/.
@ -88,6 +96,7 @@ Do you want to continue? [Y/n]
```
### `uninstall distribution`
Uninstalls an installed Enso distribution from the installation location
described in
[Installed Enso Distribution Layout](./distribution.md#installed-enso-distribution-layout).
@ -95,6 +104,7 @@ It removes the universal launcher and all components. By default, it asks the
user for confirmation, but this can be skipped by adding a `--yes` flag.
Examples:
```
> enso uninstall distribution
This will completely uninstall Enso from ~/.local/share/enso/,
@ -104,12 +114,14 @@ Do you want to continue? [y/N]
```
### `list`
Lists all installed versions of Enso and managed GraalVM distributions.
Optional arguments are `enso` to just list Enso installations or `runtime` to
list the installed runtimes.
Examples:
```bash
> enso list
Enso 2.0.1 -> GraalVM 20.1.0-java11
@ -125,77 +137,90 @@ GraalVM 21.1.0-java14 (used by 1 Enso installation)
```
### `default`
Sets the default Enso version used outside of projects.
If run without arguments, displays currently configured `default` version.
Examples:
```bash
> enso default 2.0.1
default set to version 2.0.1
default set to version 2.0.1
> enso default
default version is 2.0.1
```
### `config`
Can be used to manage project configuration or global user configuration (if
outside a project or with the `--global` flag).
If only the config path is provided, currently configured value is printed.
Examples:
```bash
> enso config --global author.name Example User
> enso config author.name Example User
> enso config author.name
Example User
```
```
### `run`
Runs a project or an Enso script file.
Examples:
```bash
> enso run script.enso # runs the file in script mode
> enso run path/to/project1 # runs the project
> enso run # runs the current project based on current directory
```
```
### `repl`
Launches an Enso REPL. If outside a project, it uses the `default` Enso version.
If run inside a project or an optional project path is specified, the REPL is
run in the context of that project, using the version specified in its
configuration.
Examples:
```bash
> enso repl # version and context depend on current working directory
> enso repl /path/to/project # runs the REPL in context of the specified project
```
```
### `language-server`
Launches the language server for a given project.
Examples:
```bash
> enso language-server
> enso language-server
--server \
--root-id 3256d10d-45be-45b1-9ea4-7912ef4226b1 \
--path /tmp/content-root
```
### `upgrade`
Checks for updates of the launcher and downloads any new versions.
Examples:
```bash
> enso upgrade
Launcher has been upgraded to the latest (3.0.2) version.
> enso upgrade 2.0.1
Launcher has been downgraded to version 2.0.1.
```
```
### `version`
Prints the version of the installed launcher as well as the full version string
of the currently selected Enso distribution.
@ -220,19 +245,23 @@ Besides `enso version`, `enso --version` is also supported and yields the same
result.
### `help`
Print this summary of available command and their usage.
## General Options
### `--version`
Overrides the inferred (project local or `default`) version when running a
command.
### `--use-system-jvm`
Tells the launcher to use the default JVM (based on `JAVA_HOME`) instead of the
managed one. Will not work if the set-up JVM version is not GraalVM.
## JVM Options
For commands that launch an Enso component inside a JVM (`repl`, `run` and
`language-server`), additional parameters are passed to the launcher components.
Moreover, it is possible to pass parameters to the JVM that is used to launch

56
docs/distribution/launcher.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Enso Launcher
The launcher is used to run Enso commands (like the REPL, language server etc.)
and seamlessly manage Enso versions. This document describes it's features. Its
command-line interface is described in the [CLI](./launcher-cli.md) document.
@ -36,14 +37,16 @@ command-line interface is described in the [CLI](./launcher-cli.md) document.
<!-- /MarkdownTOC -->
## Launcher Distribution
The launcher is distributed as a native binary for each platform (Windows,
Linux, macOS). It is distributed in a ZIP archive as described in
[Enso Distribution Layout](./distribution.md#enso-distribution-layout) in two
flavors - as packages containing just the launcher binary that can then
download and install desired versions of the engine and as bundles that already
contain the latest version of Enso engine and Graal runtime corresponding to it.
flavors - as packages containing just the launcher binary that can then download
and install desired versions of the engine and as bundles that already contain
the latest version of Enso engine and Graal runtime corresponding to it.
### Using Multiple Launcher Versions Side-By-Side
Multiple portable distributions of the launcher can be used side-by-side. To use
multiple installed distributions, some tricks are necessary - before launching a
different version, the environment variables `ENSO_DATA_DIRECTORY`,
@ -51,6 +54,7 @@ different version, the environment variables `ENSO_DATA_DIRECTORY`,
corresponding to that version.
### Detecting Portable Distribution
As described in
[Enso Distribution Layout](./distribution.md#enso-distribution-layout), the
launcher can either be run in a portable distribution or installed locally. The
@ -60,12 +64,14 @@ the parent directory for a file called `.enso.portable`. If such file is found,
the launcher runs in portable mode. Otherwise, it runs in installed mode.
## Launcher Build
The launcher is built using
[GraalVM Native Image](https://www.graalvm.org/docs/reference-manual/native-image/)
which compiles the JVM code into a native binary ahead of time, resulting in a
small and fast launching executable.
### Portability
On Linux, it is possible to statically link all libraries required by the Native
Image, thus ensuring portability between Linux distributions.
@ -75,14 +81,17 @@ generally compatible between distribution versions on these platforms.
Non-system dependencies are included in the binary on these platforms as well.
## Project Management
The launcher provides basic project management utilities for the command-line
user.
### Creating a Project
It allows to create an empty project in a specified directory with the basic
configuration based on user's config.
### Per-Project Enso Version
Project configuration can specify the exact Enso version that should be used
inside that project. The launcher automatically detects if it is in a project
(by traversing the directory structure). The current project can also be
@ -91,16 +100,18 @@ use the version specified in the project configuration, or outside a project,
the default version.
> The actionables for this section are:
>
>
> - Decide how to support inexact project bounds (like `>=3.1, <4` - when should
> the launcher check for new versions) and resolvers.
> - Decide how to support nightly builds.
### Project Configuration
The command-line allows to edit project configuration, for example: change the
author's name or Enso version.
## Enso and Graal Version Management
The launcher automatically manages required Enso versions. When running inside a
project, if the version specified in project configuration is not installed, it
is installed automatically.
@ -115,6 +126,7 @@ launching this version of Enso. When a managed GraalVM distribution is no longer
used by any installed version of Enso, it is automatically removed.
### GraalVM Override
While the launcher manages its own installation of GraalVM to ensure that the
right JVM version is used to launch each version of Enso, the user can override
this mechanism to use the installed system JVM instead. This is an advanced
@ -125,6 +137,7 @@ GraalVM distribution. It will also print a warning if the major or minor version
is different then required by that particular Enso version.
### Downloading Enso Releases
The releases are discovered and downloaded using the
[GitHub API](https://docs.github.com/en/rest/reference/repos#releases). As
described in the [Release Policy](./release-policy.md#github-releases), each
@ -142,33 +155,35 @@ ignored by the launcher unless it is specified by an exact version match. In
that case it is downloaded, but a warning is printed.
### Downloading GraalVM Releases
GraalVM is downloaded from its
[GitHub releases page](https://github.com/graalvm/graalvm-ce-builds/releases)
using GitHub API, similarly as Enso releases.
## Running Enso Components
The primary purpose of the launcher is running various Enso components, namely
the REPL, running a project, Enso scripts or the language server.
The launcher automatically infers which Enso version to used, based on the
parameters and configuration:
- When running a project or the language server, the version specified in
project configuration is used.
- When running the REPL, if the current directory is inside a project,
the project version is used, otherwise the `default` version from the global
configuration is used. The current path is the working directory unless
overridden with the `--path` parameter.
- When running an Enso script, if that script is located inside a project, the
project version is used, if it is outside a project, the `default` version is
used.
- When running a project or the language server, the version specified in
project configuration is used.
- When running the REPL, if the current directory is inside a project, the
project version is used, otherwise the `default` version from the global
configuration is used. The current path is the working directory unless
overridden with the `--path` parameter.
- When running an Enso script, if that script is located inside a project, the
project version is used, if it is outside a project, the `default` version is
used.
Additional arguments passed to the launcher are forwarded to the launched
component. Moreover, options for the JVM that is used to run the components
can also be provided, as described in
[JVM Options](./launcher-cli.md#jvm-options).
component. Moreover, options for the JVM that is used to run the components can
also be provided, as described in [JVM Options](./launcher-cli.md#jvm-options).
### Running Plugins
If the launcher gets an unknown command `foo`, it tries to run `enso-foo` and
pass all the arguments that follow. If `enso-foo` is not found, it fails as
normal. This can be used to implement plugins that are launched through the
@ -182,6 +197,7 @@ supported. That description will be included in the command listing printed by
`enso help`.
## Global User Configuration
The launcher allows to edit global user configuration, saved in the `config`
directory inside the Enso distribution structure.
@ -190,11 +206,13 @@ and used for creating new projects. It also specifies default project metadata
used when creating a project with the `new` command.
## Updating the Launcher
Besides managing Enso versions, the launcher has the ability to also update
itself. By default it is updated to the latest version available, but it also
allows downgrades by specifying a version explicitly.
### Minimal Required Launcher Version
Each version of Enso can specify the minimum version of launcher required to run
it. This version is specified in a
[manifest file](./release-policy.md#manifest-file) that should be included as an
@ -207,8 +225,9 @@ minimal required launcher version.
If the launcher detects that the installed version is older than one of the two
criteria above, it asks the user to upgrade the launcher using the `upgrade`
command.
### Downloading Launcher Releases
The launcher is released alongside Enso, so each new release of Enso also
contains native artifacts for the launcher for each platform. They are
downloaded in the same way as Enso distribution. The launcher does not have to
@ -217,6 +236,7 @@ a new Enso version requires a more recent launcher or the user explicitly wants
to.
#### Fallback Method
To ensure that the launcher can be safely updated even if the distribution
scheme changes, there should be support for a fallback upgrade scheme that is
used if the default upgrade process fails.
@ -233,6 +253,6 @@ distributed on the new distribution scheme).
Thus, when migrating to a new distribution scheme, old versions should also be
preserved, but the fallback upgrade scheme does not have to keep track of all
the versions, but only the latest one.
It can be implemented by uploading the most recent artifacts to some fixed
domain, like `launcherupgrade.release.enso.org`.

9
docs/distribution/licences.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Licenses
When distributing Enso, we include code from many dependencies that are used
within it. We need to ensure that we comply with the licenses of the
dependencies that we distribute with Enso.
@ -23,10 +24,12 @@ dependencies that we distribute with Enso.
<!-- /MarkdownTOC -->
## Gathering Used Dependencies
As a first step, we need to gather a list of which dependencies are used in the
distributed artifacts.
### SBT
We can use the plugin `sbt-license-report` to gather a list of used dependencies
and their licences. To use it, run `enso/dumpLicenseReport` in the SBT shell.
This will gather dependency information for all the subprojects. For each
@ -40,6 +43,7 @@ actually built into distributable artifacts. For now these are: `runtime`,
`runner`, `project-manager` and `launcher`.
#### `sbt-license-report` Configuration
Settings for the plugin are defined in the `licenseSettings` variable in
[`build.sbt`](../../build.sbt). The settings have to be applied to each project
by adding `.settings(licenseSettings)` to the project definition (defining these
@ -53,6 +57,7 @@ Currently it is set to only consider `compile` dependencies, as dependencies for
`provided`, `test` or `benchmark` are not distributed.
### Rust
We do not distribute any Rust-based artifacts in this repository.
> The actionables for this section are:
@ -62,6 +67,7 @@ We do not distribute any Rust-based artifacts in this repository.
> gathering dependencies used in the Rust projects.
## Preparing the Distribution
When a new dependency is added, its transitive dependencies have to be analysed
as described in the previous section. Various action has to be taken depending
on the particular licences.
@ -74,6 +80,7 @@ the distribution by adding them to an aggregate `NOTICE` file. To find these
dependencies or visit project websites of each dependency.
### Launcher
As the launcher is distributed as a native binary executable, the licences and
notices have to be included separately.
@ -84,6 +91,7 @@ directory. The notices should be gathered in
files are included by the CI build within the built artifacts.
### Engine Components
> The actionables for this section are:
>
> - The engine components as distributed as a JAR archive that everyone can
@ -92,6 +100,7 @@ files are included by the CI build within the built artifacts.
> notices should be replicated in the distributed packages anyway.
### Bundles
Beside the launcher and engine components, the distributed bundles also contain
a distribution of GraalVM CE. This distribution however contains its own licence
and notices within itself, so no further action should be necessary in that

90
docs/distribution/packaging.md
View File

@ -7,10 +7,11 @@ order: 2
---
# Enso Libraries Packaging
Given the nature of Enso as an open-source programming language and platform,
it is crucial that we provide users with an extensible package management
system. This document describes the current state of our packaging efforts, as
well as future directions and enhancements to it.
Given the nature of Enso as an open-source programming language and platform, it
is crucial that we provide users with an extensible package management system.
This document describes the current state of our packaging efforts, as well as
future directions and enhancements to it.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -23,6 +24,7 @@ well as future directions and enhancements to it.
<!-- /MarkdownTOC -->
## Enso Package Structure
The general directory structure of an Enso package is as follows:
```
@ -41,12 +43,13 @@ My_Package
```
### The `src` Directory
The `src` directory contains all Enso sources, organized in a hierarchical
structure. The structure of this directory dictates how particular modules
are imported in all of Enso code.
Note that all files and directories in this subtree must be named according
to the Enso style for referent names (i.e. `Upper_Snake_Case`, see
The `src` directory contains all Enso sources, organized in a hierarchical
structure. The structure of this directory dictates how particular modules are
imported in all of Enso code.
Note that all files and directories in this subtree must be named according to
the Enso style for referent names (i.e. `Upper_Snake_Case`, see
[the syntax specification](../syntax/naming.md#naming-constructs)).
A file located at the path `My_Package/src/Sub_Module/Helper.enso` will be
@ -59,8 +62,8 @@ import My_Package.Sub_Module.Helper
Please note the following:
- The name of the package appears as the first segment of the name.
- The package name is not specified by the containing directory's name,
but rather it is described in the `package.yaml` file.
- The package name is not specified by the containing directory's name, but
rather it is described in the `package.yaml` file.
The exact transformation is as follows:
@ -68,18 +71,20 @@ The exact transformation is as follows:
name.
2. Any subdirectories on the path from the `src` directory to the source file
are appended as consecutive segments.
3. The name of the source file, with the `.enso` extension stripped, becomes
the last segment.
3. The name of the source file, with the `.enso` extension stripped, becomes the
last segment.
### The `polyglot` Directory
The `polyglot` directory contains per-language subdirectories containing files
used by the supported polyglot languages. The contents of each subdirectory is
specified on a per-language basis, in the
[polyglot documentation](../polyglot/README.md).
### The `package.yaml` File
`package.yaml` describes certain package metadata, such as its name, authors
and version. It also includes the list of extra dependencies of the package
`package.yaml` describes certain package metadata, such as its name, authors and
version. It also includes the list of extra dependencies of the package
(dependencies that are not present in the resolver or need a version override).
The following is an example of this manifest file.
@ -91,53 +96,57 @@ author: "John Doe <john.doe@example.com>"
maintainer: "Jane Doe <jane.doe@example.com>"
resolver: lts-1.2.0
extra-dependencies:
- name: Base
version: "1.2.0"
- name: Http
version: "4.5.3"
- name: Base
version: "1.2.0"
- name: Http
version: "4.5.3"
```
The following is the specification of the manifest fields.
Fields marked as **Optional (required for publishing)** are completely optional
during development - if not specified, their default values will be used.
However, they must be specified before publishing the package. A package missing
any of these fields cannot be published.
The following is the specification of the manifest fields. Fields marked as
**Optional (required for publishing)** are completely optional during
development - if not specified, their default values will be used. However, they
must be specified before publishing the package. A package missing any of these
fields cannot be published.
#### license
**Optional (required for publishing)** *String*: The short license name of this
**Optional (required for publishing)** _String_: The short license name of this
package. Defaults to `None`, meaning the package is not safe for use by third
parties.
#### version
**Optional (required for publishing)** *String*: The
**Optional (required for publishing)** _String_: The
[semantic versioning](https://semver.org/) string, in the `major.minor.patch`
format. If not set, it defaults to `dev` (which can be used for development, but
is not a valid version for publishing).
#### author
**Optional** *String* or *List of Strings*: The name(s) and contact info(s) of
**Optional** _String_ or _List of Strings_: The name(s) and contact info(s) of
the author(s) of this library, in the `Name <contact>` or `Name` format.
#### maintainer
**Optional** *String* or *List of Strings*: The name(s) and contact info(s)
of the current maintainer(s) of this library, in the `Name <contact>` or `Name`
**Optional** _String_ or _List of Strings_: The name(s) and contact info(s) of
the current maintainer(s) of this library, in the `Name <contact>` or `Name`
format.
#### resolver
**Note** This field is not currently implemented.
**Optional (required for publishing)** *String*: The resolver name, used to
choose compiler version and basic libraries set. If not set, the system-default
resolver will be used.
**Note** This field is not currently implemented. **Optional (required for
publishing)** _String_: The resolver name, used to choose compiler version and
basic libraries set. If not set, the system-default resolver will be used.
> The actionables for this section are:
>
> - Extend the compiler version to handle version bounds.
#### extra-dependencies
**Note** This field is not currently implemented.
**Optional** *List of Library objects*: The list of libraries this package
requires to function properly and that are not included in the resolver.
Defaults to an empty list.
**Note** This field is not currently implemented. **Optional** _List of Library
objects_: The list of libraries this package requires to function properly and
that are not included in the resolver. Defaults to an empty list.
A library object is of the form:
@ -151,13 +160,14 @@ version: <semver string of the required library version>
> - Extend the library version field to handle version bounds.
## Build Reproducibility
It is crucial for any good development environment to provide reproducible
builds, such that it is impossible for it to go wrong by mismatching library
versions.
> The actionables for this section are:
>
> - Decide on the strategies of ensuring consistent library resolution. This
> may include hashing the downloaded versions of libraries and publishing
>
> - Decide on the strategies of ensuring consistent library resolution. This may
> include hashing the downloaded versions of libraries and publishing
> stack-style resolvers for sets of libraries that are proven to work well
> together.

20
docs/distribution/release-policy.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Release Policy
As an open-source project and programming language, it is incredibly important
that we have a well-defined release policy. This document defines said policy.
@ -26,6 +27,7 @@ that we have a well-defined release policy. This document defines said policy.
<!-- /MarkdownTOC -->
## Versioning
Releases of Enso are versioned using [semantic versioning](https://semver.org).
Where `a.b.c-tag` is the version string, `a` is the major version, `b`, is the
minor version, `c` is the patch version, and `tag` is additional metadata, the
@ -40,10 +42,12 @@ following hold:
pre-release change is made. These are not intended to be stable.
### Launcher Versioning
The launcher is released alongside Enso releases, so the launcher version is
tied to the Enso version that it is released with.
## Release Branches
A release branch in the Enso repository is a branch prefixed with `release/`.
Release branches obey the following rules:
@ -59,6 +63,7 @@ It should be noted that general development still takes place on the `main`
branch of the repository.
## Release Workflow
Cutting a release for Enso proceeds as follows:
1. If no release branch exists for the current major version, one should be
@ -76,16 +81,19 @@ Cutting a release for Enso proceeds as follows:
official.
### Tag Naming
Tags for releases are named as follows `enso-version`, where `version` is the
semver string (see [versioning](#versioning)) representing the version being
released.
### GitHub Releases
A release is considered _official_ once it has been made into a release on
[GitHub](https://github.com/enso-org/enso/releases). Once official, a release
may not be changed in any way, except to mark it as broken.
#### Manifest File
Each GitHub release contains an asset named `manifest.yaml` which is a YAML file
containing metadata regarding the release. The manifest is also included in the
root of an Enso version package. It has at least the following fields:
@ -98,6 +106,7 @@ root of an Enso version package. It has at least the following fields:
different Java versions, this specifies which variant to use.
For example:
```yaml
minimum-launcher-version: 0.0.1
graal-vm-version: 20.1.0
@ -111,6 +120,7 @@ stored in
and other values are added to this template at build time.
#### Release Assets Structure
Each release contains a build of the Enso engine and native launcher binaries
for each supported platform. Moreover, for convenience, it should include
bundles containing native launcher binaries and the latest engine build for each
@ -126,6 +136,7 @@ platform. So each release should contain the following assets:
- `manifest.yaml`
#### Marking a Release as Broken
We intend to _never_ delete a release from GitHub, as users may have projects
that depend on specific versions of Enso. Instead, we provide a mechanism for
marking releases as broken that works as follows:
@ -133,12 +144,13 @@ marking releases as broken that works as follows:
- An empty file named `broken` is uploaded to the release.
- The release description is edited to visibly mark the release as broken.
A broken release is one that _must not_ be downloaded by the launcher unless
a project specifies _an exact version match_, and it _must not_ be used in new
A broken release is one that _must not_ be downloaded by the launcher unless a
project specifies _an exact version match_, and it _must not_ be used in new
projects by the launcher unless _explicitly_ specified by the user as an exact
version match.
### Release Notes
Release notes should contain a summary of the changes made between the last
release and the current release. They should follow the template given below:
@ -181,7 +193,6 @@ release and the current release. They should follow the template given below:
- A list of changes that do not have user-facing impact, but represent
significant improvements to the internals of Enso and related tools.
```
If there are no changes for a section, the section should contain a bullet point
@ -191,18 +202,21 @@ The changelog file is an ongoing record of changes, and may diverge between
`main` and the various release branches.
## Version Support
We aim to support a given major version for some period of time after the
release of the next major version. For a detailed breakdown of the major
versions that are currently supported, please see the [security](./security.md)
document.
## Working on the Current Release
When working on the current release, development should take place against the
`main` branch. When it is time to cut a release, the new commits on the main
branch are cherry-picked onto the current release branch. From there, the
release proceeds as described in [release workflow](#release-workflow) above.
## Backporting Fixes
Supporting a major version for some time after the release of the next major
version will sometimes require backporting a fix to the previous major version
from the current version or from `main`.

43
docs/enso-philosophy.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Enso: Simplicity and Correctness
Enso is an award-winning, general-purpose, purely functional programming
language with equivalent, first-class visual and textual representations and a
highly expressive and novel type system.
@ -23,10 +24,10 @@ From a technical point of view, Enso incorporates many of the recent innovations
in the design and development of programming languages to improve the user
experience. It provides higher-order functions, strict semantics with opt-in
laziness, and opt-in algebraic data types, all under the auspices of a novel
type system that merges dependent typing with a great user experience. Enso
is the culmination of many years of research into functional programming
languages, consolidating the work trail-blazed by Haskell, Idris and Agda, but
also improvements in user-experience and ergonomics.
type system that merges dependent typing with a great user experience. Enso is
the culmination of many years of research into functional programming languages,
consolidating the work trail-blazed by Haskell, Idris and Agda, but also
improvements in user-experience and ergonomics.
All in all, Enso is a programming language unlike any other before it, a
seamless blend of code and visual communication that can span organisations.
@ -57,6 +58,7 @@ seamless blend of code and visual communication that can span organisations.
<!-- /MarkdownTOC -->
## Why a New Programming Language?
Since the 1980s, the way that programmers work and the tools that they use have
changed remarkably little. However, there is a small but growing chorus that
worries that the status quo is unsustainable: programmers are having problems
@ -72,6 +74,7 @@ same way that writing and speaking coexist, there are times where code is more
convenient than any other approach.
## Software Correctness Matters
"In September 2007, Jean Bookout was driving on the highway with her best friend
in a Toyota Camry when the accelerator seemed to get stuck. When she took her
foot off the pedal, the car didn't slow down. She tried the brakes but they
@ -119,10 +122,12 @@ correctness and provide as much valuable information to the user as possible in
as automated and intuitive a fashion as possible.
## Connections to your Data
Software creation is a very creative process. However, while using conventional
languages, programmers are like chess players trying to play with a blindfold on
- so much of their mental energy is spent just trying to picture where the
pieces are that there’s hardly any left over to think about the game itself.
pieces are that there’s hardly any left over to think about the game itself.
Enso was designed around the idea that _people need an immediate connection to
what they are making_, as described by by Brett Victor in his amazing talk
@ -141,6 +146,7 @@ environment for the modelling of data flows, with extensive inbuilt capabilities
for data visualisation and manipulation.
## Tenets of Enso
The design of a tool can have a _drastic_ effect on the
[cognitive load](https://en.wikipedia.org/wiki/Cognitive_load) experienced by
users. As we design Enso, we rely on a small set of principles to guide the
@ -177,10 +183,11 @@ the cognitive load for users. They are elucidated below
possible.
### Visual and Textual
While Enso _is_ a programming language, the visual environment with which it
ships is just as important. This means that when designing the language, and
any new feature for the language, we need to explicitly account for how it works
in both the visual and textual syntaxes.
ships is just as important. This means that when designing the language, and any
new feature for the language, we need to explicitly account for how it works in
both the visual and textual syntaxes.
- **Necessity:** Is this functionality necessary in both environments, or is it
compensating for a deficiency in one environment.
@ -194,6 +201,7 @@ so any functionality from one syntax that doesn't fit in the other should be
rejected.
### Unified Syntax
With the syntax being the primary mode of interaction between users and Enso,
whether it be visual or textual, it is important that it is consistent within
itself. This is quite a nebulous concept, but it boils down to a few main
@ -214,6 +222,7 @@ to be remembered, so it is incredibly important to find a happy middle ground
between _too many_ and _not enough_ syntactic constructs.
### Visual Communication
Humans are inherently visual creatures, and Enso is designed to enable a
_visual_ style of communication. When designing new language features, it is
overwhelmingly important to consider their impact on Enso's visualisation
@ -223,6 +232,7 @@ functionality.
- _How_ they can be visualised themselves.
### One Right Way
Choice in a language inherently creates myriad ways to do things. This is
singularly unhelpful in any programming language, let alone one intended to span
all experience levels, as it increases the cognitive burden of using the
@ -239,18 +249,20 @@ weighted, and a decision made based on the balance between complexity and
ergonomics.
### Help the User
Features with significant usability benefits for users should be weighted very
highly in Enso's design. This is true _even_ when adding such a feature may
bring significant complexity to the implementation of the language or tooling,
as the user experience is paramount.
That is not to say, however, that _any and all_ complexity should be accepted
in the aid of user experience. If the maintenance burden is too great, then it
is likely that introducing such a photo will impact future UX improvements. To
this end, it is always a careful balancing act, even if we tilt the scales
heavily in favour of UX over simplicity of the implementation.
That is not to say, however, that _any and all_ complexity should be accepted in
the aid of user experience. If the maintenance burden is too great, then it is
likely that introducing such a photo will impact future UX improvements. To this
end, it is always a careful balancing act, even if we tilt the scales heavily in
favour of UX over simplicity of the implementation.
### Simple Complexity
A big part of Enso's design is the notion of employing sophisticated and
cutting-edge techniques in programming language design. However, many languages
that employ these techniques do so to the _detriment_ of the user experience.
@ -266,6 +278,7 @@ Enso should provide a good user experience, it should also not stand in the way
of power users.
### Powerful When Necessary
While we want to provide a simple tool to our users, this should not induce the
language to shy away from necessary power.
@ -276,6 +289,7 @@ should be opt-in, with users not wanting the enhancements to expressiveness not
having to know or care about it.
### Performance and Predictability
There is little that is more frustrating than the code you write running slowly
for no explicable reason.
@ -287,6 +301,7 @@ aim for a sensible mapping from source code to machine execution, and aim to
ensure that seemingly innocuous changes do not impact Enso's performance.
### Explicit Over Implicit
When designing Enso and its libraries, we don't want to have any behaviour of a
function that is not recorded in its type, or its defaults. This gives rise to
two main principles for designing Enso's APIs:
@ -335,6 +350,7 @@ reading and writing, can call it as follows: `openHandle path (fileMode = RWA)`,
or just `openHandle path RWA`.
### Guidance Over On-Boarding
While it is important to have an on-boarding process to help bring new users up
to speed, that process cannot be the only guidance that users of Enso get.
@ -348,6 +364,7 @@ In essence, it means designing for the _enduring_ user experience of Enso,
rather than purely the initial user experience.
## Designing Enso
As Enso's design and functionality evolves, we have to take the utmost care to
ensure that it doesn't balloon beyond control. As a result, every new feature
that we contemplate adding to the language should advance these core tenets, and

17
docs/getting-enso.md
View File

@ -7,8 +7,9 @@ order: 5
---
# Getting Enso
Enso packages can currently be obtained from the per-commit CI builds.
See [the build workflow on GitHub Actions](https://github.com/enso-org/enso/actions?query=workflow%3A%22Engine+CI%22).
Enso packages can currently be obtained from the per-commit CI builds. See
[the build workflow on GitHub Actions](https://github.com/enso-org/enso/actions?query=workflow%3A%22Engine+CI%22).
The artifact of interest is `enso-<version>` (currently `enso-0.0.1`).
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -20,18 +21,20 @@ The artifact of interest is `enso-<version>` (currently `enso-0.0.1`).
<!-- /MarkdownTOC -->
## Dependencies
The Enso distribution requires to be run with the appropriate version of
GraalVM. You can get the Community Edition pre-built distributions from
[the GitHub releases site](https://github.com/graalvm/graalvm-ce-builds/releases).
It is important to run Enso with exactly the version specified here. Given that
Graal is still a relatively young project, even the minor version changes
introduce breaking API changes. The current version of GraalVM required for
Enso is `20.1.0`, and it must be the Java 11 build.
introduce breaking API changes. The current version of GraalVM required for Enso
is `20.1.0`, and it must be the Java 11 build.
Before running the Enso packages, make sure that the `JAVA_HOME` environment
variable points to the correct home location of the Graal distribution.
## Running Enso
The distribution contains two main executables of interest:
1. The project manager. This executable is currently used for testing the IDE,
@ -46,18 +49,22 @@ The distribution contains two main executables of interest:
Again, it is necessary for you to set the `JAVA_HOME` variable correctly.
## Troubleshooting
This section lists the most common failures and their probable causes.
1. The exception `java.lang.IllegalArgumentException: Could not find option with name enso-runtime-server.enable.`
1. The exception
`java.lang.IllegalArgumentException: Could not find option with name enso-runtime-server.enable.`
It can contain a different option name. This exception signals problems with
the Graal distribution. Make sure you're running Enso with the correct
version of GraalVM. You can verify the version of JDK used by running
`bin/enso --version`. Take note of the version displayed in the `Running on`
section. It should be similar to:
```
Running on: OpenJDK 64-Bit Server VM, GraalVM Community, JDK 11.0.7+10-jvmci-20.1-b02
Linux 4.15.0-106-generic (amd64)
```
It could also be caused by not using the launcher scripts and trying to run
the component `.jar` files via `java -jar` without setting the necessary
options. Use the launcher scripts.

1
docs/infrastructure/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Infrastructure
The Enso runtime runs on the GraalVM which is a version of the JVM. This folder
contains all documentation pertaining to Enso's infrastructure, which is broken
up as follows:

18
docs/infrastructure/java-11.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Migrating to Java 11
JDK 11 will be supported longer than JDK 8 that we currently use and it adds new
features that could improve performance. Moreover, we want to be compliant to
the Java Platform Module System, as all future versions of the JDK will rely on
@ -24,21 +25,24 @@ it. Thus, we have moved to using Graal builds for Java 11.
<!-- /MarkdownTOC -->
## Migration Progress
The overall steps of the migration and their status are outlined in this
section.
### Build Configuration
The option `-XX:-UseJVMCIClassLoader` is deprecated in Java 11 and has been
removed from the test configuration.
removed from the test configuration.
The JVM running sbt must have `--upgrade-module-path=lib/truffle-api.jar` added
as an option and the build tool must ensure that the `truffle-api.jar` is copied
from the Maven repository to the `lib/` directory before the `runtime` project
is compiled. Section [IllegalAccessError](#illegalaccesserror) explains why
this is necessary and [Bootstrapping](./sbt.md#bootstrapping) explains the tasks
that help with this process.
is compiled. Section [IllegalAccessError](#illegalaccesserror) explains why this
is necessary and [Bootstrapping](./sbt.md#bootstrapping) explains the tasks that
help with this process.
### Testing
All tests are passing.
To make sure `runtime` tests can be run both with `test` and `testOnly`,
@ -47,6 +51,7 @@ To make sure `runtime` tests can be run both with `test` and `testOnly`,
directory different than the project root, so an absolute path should be used.
### Benchmarks
Initially there were some regressions found in the benchmarks, but further
investigation revealed this was caused by some issues in the methodology of how
the JMH benchmarks were implemented. There are plans to rewrite these
@ -56,9 +61,11 @@ Benchmarks in pure Enso are currently more meaningful. They yield comparable
results with Java 11 being slightly faster.
## Problems
The problems that were encountered when doing the migration.
### IllegalAccessError
As described in [Build tools](sbt.md#incremental-compilation), to allow for
incremental compilation, zinc has to analyse dependencies between various files.
As our project uses both Scala and Java code, dependencies between files in both
@ -86,9 +93,10 @@ printed. Others are not detected where zinc expects them, but they fail later,
crashing the compilation process. All of them are problematic though, because
they mean that zinc is not able to read dependencies of the affected files. This
harms incremental compilation (as some dependencies are not detected it might be
necessary to do a clean and full recompilation when changing these files).
necessary to do a clean and full recompilation when changing these files).
#### Solution
We want to make the ClassLoader read these class files without errors. For that
we need to ensure it has permissions to load the Truffle modules.

24
docs/infrastructure/sbt.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Build Tools
The project is built using the Scala Build Tool which manages dependencies
between the projects as well as external dependencies and allows for incremental
compilation. The build configuration is defined in
@ -29,6 +30,7 @@ compilation. The build configuration is defined in
<!-- /MarkdownTOC -->
## Incremental Compilation
To help wit build times, we do not want to rebuild the whole project with every
change, but to only recompile the files that have been affected by the change.
This is handled by sbt which under the hood uses
@ -37,6 +39,7 @@ analyses the compiled files and detects dependencies between them to determine
which files have to be recompiled when something has been changed.
## Bootstrapping
As described in [Java 11 Migration](./java-11.md#illegalaccesserror) to
successfully compile the `runtime` project, the JVM running sbt must use the
overridden JAR for Truffle API. This JAR has to be present during startup of the
@ -54,6 +57,7 @@ The logic for copying the JAR is implemented in the `bootstrapJARs` task in
[`CopyTruffleJAR`](../../project/CopyTruffleJAR.scala).
## Compile Hooks
There are some invariants that are specific to our project, so they are not
tracked by sbt, but we want to ensure that they hold to avoid cryptic errors at
compilation or runtime.
@ -74,21 +78,23 @@ the task `compileIncremental`. `Compile / compile` depends on
`compileIncremental` but if we add our dependency to `Compile / compile`, it is
considered as independent with `compileIncremental`, so sbt may schedule it to
run in parallel with the actual compilation process. To guarantee that our
pre-flight checks complete *before* the actual compilation, we add them as a
dependency of `compileInputs` which runs *strictly before* actual compilation.
pre-flight checks complete _before_ the actual compilation, we add them as a
dependency of `compileInputs` which runs _strictly before_ actual compilation.
To check some invariants *after* compilation, we can replace the original
To check some invariants _after_ compilation, we can replace the original
`Compile / compile` task with a custom one which does its post-compile checks
and returns the result of `(Compile / compile).value`. An example of such a
'patched' compile task is implemented in
[`FixInstrumentsGeneration`](../../project/FixInstrumentsGeneration.scala).
## Helper Tasks
There are additional tasks defined in the [`project`](../../project) directory.
They are used by [`build.sbt`](../../build.sbt) to provide some additional
functionality.
### Graal and Flatc Version Check
[`EnvironmentCheck`](../../project/EnvironmentCheck.scala) defines a helper
function that can be attached to the default `Global / onLoad` state transition
to run a version check when loading the sbt project. This helper function
@ -99,15 +105,18 @@ system with the Flatbuffers library version defined in
telling the user to change to the correct version.
### Benchmarks
[`BenchTasks`](../../project/BenchTasks.scala) defines configuration keys for
benchmarking.
### Build Information
[`BenchTasks`](../../project/BuildInfo.scala) records version information
including what git commit has been used for compiling the project. This
information is used by `enso --version`.
### Instruments Generation
Truffle annotation processor generates a file that registers instruments
provided by the runtime. Unfortunately, with incremental compilation, only the
changed instruments are recompiled and the annotation processor does not detect
@ -119,7 +128,7 @@ detects changes to instruments and if only one of them should be recompiled, it
forces recompilation of all of them, to ensure consistency.
For unclear reasons, if this task is attached to
`Compile / compile / compileInputs`, while it runs strictly *before*
`Compile / compile / compileInputs`, while it runs strictly _before_
compilation, the deleted class files are not always all recompiled. So instead,
it is attached directly to `Compile / compile`. This technically could allow for
a data race between this task and the actual compilation that happens in
@ -133,6 +142,7 @@ consistency it ensures the instruments will be recompiled the next time and
stops the current compilation task, asking the user to restart it.
### Flatbuffers Generation
[`GenerateFlatbuffers`](../../project/GenerateFlatbuffers.scala) defines the
task that runs the Flatbuffer compiler `flatc` whenever the flatbuffer
definitions have been changed. It also makes sure that `flatc` is available on
@ -140,6 +150,7 @@ PATH and that its version matches the version of the library. It reports any
errors.
### Ensuring JARs Were Loaded
As described in [Bootstrapping](#bootstrapping), to successfully compile the
`runtime` subproject, the JVM running sbt must load some JARs at startup. The
user should run `sbt bootstrap` to ensure that.
@ -147,7 +158,7 @@ user should run `sbt bootstrap` to ensure that.
If the compilation proceeds without the bootstrapped JARs it may lead to
inconsistent state with some dependencies being undetected and weird errors. To
avoid such situations, [`CopyTruffleJAR`](../../project/CopyTruffleJAR.scala)
defines is a pre-compile task that is executed before compiling the `runtime`
defines is a pre-compile task that is executed before compiling the `runtime`
subproject which makes sure that the Truffle JAR is up-to-date. Even if the
developer forgets about `bootstrap`, this pre-compile task will update the
Truffle JARs. However, as the JARs are loaded at startup, the JVM has to be
@ -165,16 +176,19 @@ experience. In normal operation, the restart should never be triggered, as the
user should remember to run `bootstrap` when necessary.
### Debugging Command
[`WithDebugCommand`](../../project/WithDebugCommand.scala) defines a command
that allows to run a task with additional JVM-level flags.
### Recompile Parser
[`RecompileParser`](../../project/RecompileParser.scala) defines a task that can
be attached to the `compile` task in configurations of the `syntax` project.
This task ensures that the `syntax` project is recompiled whenever
`syntax-definition` changes.
## Native Image
[`NativeImage`](../../project/NativeImage.scala) defines a task that can compile
a project into a native binary using Graal's Native Image. It compiles the
project and runs the Native Image tool which builds the image. To be able to use

5
docs/language-server/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Language Server
The Enso Language Server is responsible for providing language services to the
Enso IDE (and other clients). This mainly involves speaking the Enso protocol
and orchestrating the runtime in response to this. It is responsible for:
@ -15,8 +16,8 @@ and orchestrating the runtime in response to this. It is responsible for:
about their running code including values, types, profiling information, and
debugging.
- **Code Execution:** The ability for clients to execute arbitrary Enso code in
arbitrary scopes. This can be used in conjunction with the above to provide
a REPL with an integrated debugger.
arbitrary scopes. This can be used in conjunction with the above to provide a
REPL with an integrated debugger.
- **Code Completion:** Sophisticated completion functionality that refines
suggestions intelligently based on context.
- **Node Management:** Tracking and providing the language server's internal

60
docs/language-server/protocol-architecture.md
View File

@ -7,6 +7,7 @@ order: 1
---
# The Enso Protocol
Enso is a sophisticated language, but in order to provide a great user
experience to our users we also need the ability to provide great tooling. This
tooling means a language server, but it also means a set of extra peripheral
@ -53,6 +54,7 @@ please see [the protocol message specifications](./README.md).
<!-- /MarkdownTOC -->
## Architecture
The divisions of responsibility between the backend engine services are dictated
purely by necessity. As multi-client editing necessitates careful
synchronisation and conflict resolution, between the actions of multiple
@ -73,6 +75,7 @@ decision as to run them in different processes until the requirements become
more clear.
### The Project Manager
The project manager service is responsible for both allowing users to work with
their projects but also the setup and teardown of the language server itself.
Its responsibilities can be summarised as follows:
@ -83,6 +86,7 @@ Its responsibilities can be summarised as follows:
it to persist any state that it needs to disk.
### Language Server
The language server is responsible for managing incoming connections and
communicating with the clients, as well as resolving any potential conflicts
between the clients. It is responsible for the following:
@ -123,6 +127,7 @@ language server. Instead, the LS should only depend on `org.graalvm.polyglot` to
interface with the runtime.
## Textual Protocol
The protocol refers to the communication format that all of the above services
speak between each other and to the GUI. This protocol is not specialised only
to language server operations, as instead it needs to work for all of the
@ -140,13 +145,14 @@ The protocol we are using intends to be fully compatible with the Microsoft LSP
commands) we will do so.
2. If this is not possible, we will specify an _extension_ to the protocol.
This extension will be well-specified within this document, and should be
in the spirit of the existing protocol. If relevant, we may propose it as
a future extension to the specification.
in the spirit of the existing protocol. If relevant, we may propose it as a
future extension to the specification.
Aside from the language server protocol-based operations, we will definitely
need a protocol extension to support Enso's custom language functionality.
### Textual Protocol Communication Patterns
Whatever protocol we decide on will need to have support for a couple of main
communication patterns:
@ -154,9 +160,9 @@ communication patterns:
able to support this kind of connection to deal with events that do not occur
strictly in response to client actions (e.g. updates to observed values).
- **Req/Res:** A standard request/response model, the server will need to be
able to support this kind of connection to deal with one-off requests from
the client, and potentially to make requests to the client (e.g. list modules
in the current project, please refresh your file state).
able to support this kind of connection to deal with one-off requests from the
client, and potentially to make requests to the client (e.g. list modules in
the current project, please refresh your file state).
There are also certain messages that follow the request/response model but where
the responses are trivial acknowledgements. For simplicity's sake these are
@ -173,6 +179,7 @@ We can support additional patterns through LSP's mechanisms:
- Protocol-level acknowledgements is supported directly in LSP.
### Textual Protocol Transport
The transport of the protocol refers to the underlying layer over which its
messages (discussed in [the protocol format](#the-protocol-format) below) are
sent. As we are maintaining compatibility with LSP, the protocol transport
@ -192,6 +199,7 @@ format is already defined for us.
> and tear it down.
### The Protocol Format
Protocol messages are defined by LSP. Any extensions to the messages defined in
the standard should use similar patterns such that they are not incongruous with
LSP messages. The following notes apply:
@ -204,6 +212,7 @@ This means that we have two pipes: one is the textual WebSocket defined by LSP,
and the other is a binary WebSocket.
## Textual Protocol Functionality
This entire section deals with the _functional_ requirements placed upon the
protocol used by the engine services. These requirements are overwhelmingly
imposed by the IDE, but also include additional functionality for the future
@ -214,6 +223,7 @@ those _expected_ for the 2.0 release. Any additional functionality beyond this
milestone is described in a [dedicated section](#functionality-post-20).
### Textual Diff Management
The engine services need to support robust handling of textual diffs. This is
simply because it is the primary form of communication for synchronising source
code between the IDE and the engine. It will need to support the following
@ -252,6 +262,7 @@ The implementation is as follows:
track updates as necessary.
### Handling Multiple Clients
Multiple-client support will be implemented while remaining compatible with the
LSP specification.
@ -277,6 +288,7 @@ It will work as follows:
state, and then `applyEdit` to synchronise views of the code.
### Project State Management
One of the most important functionalities for this service set is the ability to
manage the state of a project in general. The project state refers to the whole
set of the project files and metadata and needs to support the following
@ -295,6 +307,7 @@ projects in a single engine, as this would allow users to work with multiple
related projects in a single instance of the IDE.
### File Management and Storage
The nature of LSP means that file management and storage is _not_ handled by the
language server, and is instead handled by the editor. The protocol makes a
distinction between:
@ -311,6 +324,7 @@ file operations between the IDE and that machine are handled _indepdendently_ of
the language server.
### Execution Management
The language server process will need to be able to respond to requests for
various kinds of execution of Enso code. Furthermore, it needs to be able to
respond to requests to 'listen' to the execution of various portions of code.
@ -339,11 +353,11 @@ given position in the call stack.
- A subscription may encompass multiple nodes or a single node. Information is
received for _all_ nodes covered by the provided span.
- A subscription will ensure that the client receives information on changes in:
+ Execution state (whether the node is being computed or is cached)
+ Profiling information
+ Values
+ Types
+ Where we are in the call stack (useful for recursive execution)
- Execution state (whether the node is being computed or is cached)
- Profiling information
- Values
- Types
- Where we are in the call stack (useful for recursive execution)
- Such subscriptions _must_ be accompanied by heartbeat messages in order to
allow the language server to cull unused subscriptions.
- Additionally, it will be important for each subscription to be able to
@ -351,6 +365,7 @@ given position in the call stack.
client. If unspecified this should be set to a sensible default.
#### Caching
One of the most important elements of execution management for the language
server is the ability to control and interact with the execution cache state in
the runtime.
@ -378,6 +393,7 @@ strategies, but we need to be correct.
better cache-eviction decisions.
#### Progress Reporting
In the future it will be desirable for long running computations to provide
real-time progress information (e.g. for training a neural network it would be
great to know which epoch is running).
@ -393,6 +409,7 @@ of long-running operations within the _language server_ rather than in user
code.
### Completion
The IDE needs the ability to request completions for some target point (cursor
position) in the source code. In essence, this boils down to _some_ kind of
smart completion. The completion should provide the following:
@ -441,6 +458,7 @@ From an implementation perspective, the following notes apply:
any optimisations here.
### Analysis Operations
We also want to be able to support a useful set of semantic analysis operations
to help users navigate their code. As these rely on knowledge of the language
semantics, they must be explicitly supported by the language server:
@ -451,6 +469,7 @@ semantics, they must be explicitly supported by the language server:
file is closed, then the edit should be made directly, as the LSP specifies.
### Functionality Post 2.0
In addition to the functionality discussed in detail above, there are further
augmentations that could sensibly be made to the Engine services to support a
much better editing and user-experience for Enso. These are listed briefly below
@ -479,7 +498,8 @@ and will be expanded upon as necessary in the future.
should allow for hot-reloading of code, changing of values within a live
program, and various other debugger functionality (step over, step in, step
out, continue, etc). Future debugger functionality should be based on the
standard [debug adapter protocol](https://microsoft.github.io/debug-adapter-protocol/specification).
standard
[debug adapter protocol](https://microsoft.github.io/debug-adapter-protocol/specification).
- **Profiling Information:** Profiling information for the executing code, able
to be displayed visually in Enso Studio.
- **Code Formatting:** Automatic formatting of Enso code using the One True
@ -503,6 +523,7 @@ and will be expanded upon as necessary in the future.
relevant to our language. Currently we only support a small subset thereof.
## Binary Protocol
The binary protocol refers to the auxiliary protocol used to transport raw
binary data between the engine and the client. This functionality is _entirely_
extraneous to the operation of the [textual protocol](#textual-protocol), and is
@ -585,6 +606,7 @@ table Success {}
```
### Binary Protocol Communication Patterns
The binary protocol currently only supports a single type of communication
pattern:
@ -592,6 +614,7 @@ pattern:
performed using the textual protocol.
### Binary Protocol Transport
The binary protocol uses [flatbuffers](https://github.com/google/flatbuffers)
for the protocol transport format. This choice has been made for a few reasons:
@ -601,15 +624,17 @@ for the protocol transport format. This choice has been made for a few reasons:
- Robust, schema-based messages.
## Binary Protocol Functionality
The binary protocol exists in order to serve the high-bandwidth data transfer
requirements of the engine and the GUI.
### Displaying Visualisations
A major part of Enso Studio's functionality is the rich embedded visualisations
that it supports. This means that the following functionality is necessary:
- Execution of an arbitrary Enso expression on a cached value designated by
a source location.
- Execution of an arbitrary Enso expression on a cached value designated by a
source location.
- The ability to create and destroy visualisation subscriptions with an
arbitrary piece of Enso code as the preprocessing function.
- The ability to update _existing_ subscriptions with a new preprocessing
@ -632,6 +657,7 @@ From the implementation perspective:
visualisation data to identify an update.
## Service Connection Setup
As these services need to support multiple clients in future, there is some
rigmarole around setting up the various connections needed by each client. The
process for spawning and connecting to an engine instance is as follows:
@ -642,14 +668,16 @@ process for spawning and connecting to an engine instance is as follows:
be used to identify the client while it is connected.
3. **Protocol Connection Initialisation:** The client performs the init for the
textual protocol connection, passing its client identifier as it does so.
See [`session/initProtocolConnection`](./protocol-language-server.md#sessioninitprotocolconnection)
See
[`session/initProtocolConnection`](./protocol-language-server.md#sessioninitprotocolconnection)
for more information.
4. **Data Connection Initialisation:** The client performs the init for the
data connection, passing its client identifier as it does so. See
[`session/initDataConnection`](./protocol-language-server.md#sessioninitdataconnection) below more
information.
[`session/initDataConnection`](./protocol-language-server.md#sessioninitdataconnection)
below more information.
## Service Connection Teardown
As the engine performs sophisticated caching and persisting of data where
possible, it is very important that the client informs the engine of the end of
its session. In contrast to the initialisation flow above, this is not an

13
docs/language-server/protocol-common.md
View File

@ -7,8 +7,9 @@ order: 2
---
# Enso Protocol Common Message Specification
This document contains the specification of the Enso protocol messages that are
common to multiple sub-components of the protocol.
common to multiple sub-components of the protocol.
For information on the design and architecture of the protocol, as well as its
transport formats, please look [here](./protocol-architecture).
@ -24,6 +25,7 @@ transport formats, please look [here](./protocol-architecture).
<!-- /MarkdownTOC -->
## Protocol Message Specification
The message specification for protocol messages must include the following
fields:
@ -37,8 +39,8 @@ fields:
They must also contain separate sections specifying their parameters, result (if
it has one), and any errors that may occur. These specifications should be
either in typescript or flatbuffers syntax, depending on the connection on
which the message occurs.
either in typescript or flatbuffers syntax, depending on the connection on which
the message occurs.
The capability specifications must include the following fields, as well as a
section 'Enables' stating which protocol messages are gated by the capability.
@ -48,13 +50,16 @@ section 'Enables' stating which protocol messages are gated by the capability.
capability, described using typescript type syntax.
## Common Types
There are a number of types that are shared between many of the protocol
messages. They are specified below.
### `Path`
A path is a representation of a path relative to a specified content root.
#### Format
Please note that segments can only be ordinary file names, `..` and `.` may not
be supported.
@ -81,6 +86,7 @@ table Path {
```
### `IPWithSocket`
A IPWithSocket is an endpoint for communication between machines.
#### Format
@ -93,6 +99,7 @@ interface IPWithSocket {
```
### `EnsoUUID`
An EnsoUUID is a value object containing 128-bit universally unique identifier.
#### Format

667
docs/language-server/protocol-language-server.md
View File

File diff suppressed because it is too large Load Diff

54
docs/language-server/protocol-project-manager.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Enso Protocol Project Manager Message Specification
This document contains the specification of the Enso protocol messages that
pertain to the project manager component. Please familiarise yourself with the
[common](./protocol-common.md) features of the protocol before reading this
@ -42,10 +43,12 @@ transport formats, please look [here](./protocol-architecture).
<!-- /MarkdownTOC -->
## Types
There are a number of types that are used only within the project server's
protocol messages. These are specified here.
### `ProjectMetadata`
This type represents information about a project.
#### Format
@ -59,10 +62,12 @@ interface ProjectMetadata {
```
## Project Management Operations
The primary responsibility of the project managers is to allow users to manage
their projects.
### `project/open`
This message requests that the project manager open a specified project. This
operation also includes spawning an instance of the language server open on the
specified project.
@ -90,6 +95,7 @@ interface ProjectOpenResult {
```
#### Errors
- [`ProjectNotFoundError`](#projectnotfounderror) to signal that the project
doesn't exist.
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
@ -97,6 +103,7 @@ interface ProjectOpenResult {
- [`ProjectOpenError`](#projectopenerror) to signal failures during server boot.
### `project/close`
This message requests that the project manager close a specified project. This
operation includes shutting down the language server gracefully so that it can
persist state to disk as needed.
@ -117,10 +124,12 @@ interface ProjectCloseRequest {
#### Result
```typescript
{}
{
}
```
#### Errors
- [`ProjectNotFoundError`](#projectnotfounderror) to signal that the project
doesn't exist.
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
@ -129,11 +138,12 @@ interface ProjectCloseRequest {
during language server stoppage.
- [`ProjectNotOpenError`](#projectnotopenerror) to signal cannot close a project
that is not open.
- [`ProjectOpenByOtherPeersError`](#projectopenbyotherpeerserror) to signal
that cannot close a project that is open by other clients.
- [`ProjectOpenByOtherPeersError`](#projectopenbyotherpeerserror) to signal that
cannot close a project that is open by other clients.
### `project/list`
This message requests that the project manager lists all user's projects. The
This message requests that the project manager lists all user's projects. The
list of projects is sorted by the open time.
- **Type:** Request
@ -158,10 +168,12 @@ interface ProjectListResponse {
```
#### Errors
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
underlying data store.
### `project/create`
This message requests the creation of a new project.
- **Type:** Request
@ -186,14 +198,16 @@ interface ProjectOpenResponse {
```
#### Errors
- [`ProjectNameValidationError`](#projectnamevalidationerror) to signal
validation failures.
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
underlying data store.
- [`ProjectExistsError`](#projectexistserror) to signal that the project
already exists.
- [`ProjectExistsError`](#projectexistserror) to signal that the project already
exists.
### `project/rename`
This message requests the renaming of a project.
- **Type:** Request
@ -217,18 +231,19 @@ null
```
#### Errors
- [`ProjectNameValidationError`](#projectnamevalidationerror) to signal
validation failures.
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
underlying data store.
- [`ProjectExistsError`](#projectexistserror) to signal that the project with
the provided name already exists.
- [`ServiceError`](#serviceerror) to signal that the
the operation timed out.
- [`ServiceError`](#serviceerror) to signal that the the operation timed out.
- [`LanguageServerError`](#languageservererror) to signal generic language
server failures.
### `project/delete`
This message requests the deletion of a project.
- **Type:** Request
@ -247,10 +262,12 @@ interface ProjectDeleteRequest {
#### Result
```typescript
{}
{
}
```
#### Errors
- [`ProjectDataStoreError`](#projectdatastoreerror) to signal problems with
underlying data store.
- [`ProjectNotFoundError`](#projectnotfounderror) to signal that the project
@ -258,8 +275,8 @@ interface ProjectDeleteRequest {
- [`CannotRemoveOpenProjectError`](#cannotremoveopenprojecterror) to signal that
the project cannot be removed, because is open by at least one user.
### `project/listSample`
This request lists the sample projects that are available to the user.
- **Type:** Request
@ -284,9 +301,11 @@ interface ProjectListSampleResponse {
```
#### Errors
TBC
## Language Server Management
The project manager is also responsible for managing the language server. This
means that it needs to be able to spawn the process, but also tell the process
when to shut down.
@ -297,11 +316,12 @@ when to shut down.
> relationship is going to work.
## Errors
The project manager component has its own set of errors. This section is not a
complete specification and will be updated as new errors are added.
### `ProjectNameValidationError`
Signals validation failures.
```typescript
@ -312,6 +332,7 @@ Signals validation failures.
```
### `ProjectDataStoreError`
Signals problems with underlying data store.
```typescript
@ -322,6 +343,7 @@ Signals problems with underlying data store.
```
### `ProjectExistsError`
Signals that the project already exists.
```typescript
@ -332,6 +354,7 @@ Signals that the project already exists.
```
### `ProjectNotFoundError`
Signals that the project doesn't exist.
```typescript
@ -341,8 +364,8 @@ Signals that the project doesn't exist.
}
```
### `ProjectOpenError`
Signals that the project cannot be open due to boot failures.
```typescript
@ -353,6 +376,7 @@ Signals that the project cannot be open due to boot failures.
```
### `ProjectNotOpenError`
Signals that cannot close project that is not open.
```typescript
@ -363,6 +387,7 @@ Signals that cannot close project that is not open.
```
### `ProjectOpenByOtherPeersError`
Signals that cannot close a project that is open by other clients.
```typescript
@ -373,6 +398,7 @@ Signals that cannot close a project that is open by other clients.
```
### `CannotRemoveOpenProjectError`
Signals that cannot remove open project.
```typescript
@ -383,6 +409,7 @@ Signals that cannot remove open project.
```
### `ProjectCloseError`
Signals failures during shutdown of a server.
```typescript
@ -393,6 +420,7 @@ Signals failures during shutdown of a server.
```
### `LanguageServerError`
Signals generic language server errors.
```typescript
@ -400,4 +428,4 @@ Signals generic language server errors.
"code" : 4010,
"message" : "The language server is unresponsive"
}
```
```

1
docs/parser/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso's Parser
The parser is one of the most crucial components of the Enso runtime in that
_all_ code that a user writes must be parsed. This means that a good parser is
fast, responsive, and lightweight; it shouldn't baulk at having thousands of

2
docs/parser/architecture.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Parser Architecture Overview
The Enso parser is designed in a highly modular fashion, with separate crates
responsible for the component's various responsibilities. The overall
architecture for the parser is described in this document.
@ -18,6 +19,7 @@ architecture for the parser is described in this document.
<!-- /MarkdownTOC -->
## Overall Architecture
The overall architecture of the parser subsystem can be visualised as follows.
```

2
docs/parser/ast.md
View File

@ -7,6 +7,7 @@ order: 9
---
# AST
The parser AST describes the high-level syntactic structure of Enso, as well as
containing robust and descriptive parser errors directly in the AST.
@ -17,6 +18,7 @@ containing robust and descriptive parser errors directly in the AST.
<!-- /MarkdownTOC -->
## Functionality
The parser AST needs to account for the following:
- A single `Name` type, removing the distinction between different names found

2
docs/parser/construct-resolution.md
View File

@ -7,6 +7,7 @@ order: 7
---
# Construct Resolution
Construct resolution is the process of turning the low-level AST format into the
full high-level AST format that represents both all of Enso's language
constructs and contains rich error nodes.
@ -23,6 +24,7 @@ constructs and contains rich error nodes.
> all known current use cases.
## Syntax Errors
It is very important that Enso is able to provide descriptive and useful syntax
errors to its users. Doing so requires that it has a full understanding of the
language's syntax, but also that it is designed in such a fashion that it will

37
docs/parser/flexer.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Flexer
The flexer is a finite-automata-based engine for generating lexers. Akin to
`flex` and other lexer generators, it is given a definition as a series of rules
from which it then generates code for a highly-optimised lexer.
@ -16,13 +17,14 @@ from which it then generates code for a highly-optimised lexer.
- [Pattern Description](#pattern-description)
- [State Management](#state-management)
- [Code Generation](#code-generation)
- [Automated Code Generation](#automated-code-generation)
- [Notes on Code Generation](#notes-on-code-generation)
- [Automated Code Generation](#automated-code-generation)
- [Notes on Code Generation](#notes-on-code-generation)
- [An Example](#an-example)
<!-- /MarkdownTOC -->
## Pattern Description
The definition of a lexer using the flexer library consists of a set of rules
for how to behave when matching portions of syntax. These rules behave as
follows:
@ -55,6 +57,7 @@ A `lexer.rule(...)` definition consists of the following parts:
- The code that is executed when the pattern matches.
## State Management
States in the flexer engine provide a mechanism for grouping sets of rules
together known as `State`. At any given time, only rules from the _active_ state
are considered by the lexer.
@ -65,6 +68,7 @@ are considered by the lexer.
- You can deactivate the topmost state by using `state.pop()`.
## Code Generation
The patterns in a lexer definition are used to generate a highly-efficient and
specialised lexer. This translation process works as follows:
@ -90,15 +94,16 @@ associated code is executed and the process starts over again until the input
stream has been consumed.
### Automated Code Generation
In order to avoid the lexer definition getting out of sync with its
implementation (the generated engine), it is necessary to create a separate
crate for the generated engine that has the lexer definition as one of its
In order to avoid the lexer definition getting out of sync with its
implementation (the generated engine), it is necessary to create a separate
crate for the generated engine that has the lexer definition as one of its
dependencies.
This separation enables a call to `flexer.generate_specialized_code()` in
`build.rs` (or a macro) during compilation. The output can be stored in a new
file i.e. `lexer-engine.rs` and exported from the library with
`include!("lexer-engine.rs")`. The project structure therefore appears as
This separation enables a call to `flexer.generate_specialized_code()` in
`build.rs` (or a macro) during compilation. The output can be stored in a new
file i.e. `lexer-engine.rs` and exported from the library with
`include!("lexer-engine.rs")`. The project structure therefore appears as
follows:
```
@ -116,14 +121,15 @@ follows:
- cargo.toml <-- lexer-definition is in dependencies and build-dependencies
```
With this design, `flexer.generate_specialized_code()` is going to be executed
With this design, `flexer.generate_specialized_code()` is going to be executed
on each rebuild of `lexer/generation`. Therefore, `generation` should contain
only the minimum amount of logic (i.e. tests should be in separate crate) and
its dependencies should optimally involve only such code which directly
influences the content of generated code (in order to minimize the
unnecessary calls to expensive flexer specialization).
its dependencies should optimally involve only such code which directly
influences the content of generated code (in order to minimize the unnecessary
calls to expensive flexer specialization).
### Notes on Code Generation
The following properties are likely to hold for the code generation machinery.
- The vast majority of the code generated by the flexer is going to be the same
@ -132,8 +138,9 @@ The following properties are likely to hold for the code generation machinery.
as an argument.
## An Example
The following code provides a sketchy example of the intended API for the
flexer code generation using the definition of a simple lexer.
The following code provides a sketchy example of the intended API for the flexer
code generation using the definition of a simple lexer.
```rust
use crate::prelude::*;

1
docs/parser/jvm-object-generation.md
View File

@ -7,6 +7,7 @@ order: 10
---
# JVM Object Generation
The JVM object generation phase is responsible for creating JVM-native objects
representing the parser AST from the rust-native AST. This is required to allow
the compiler and runtime to work with the AST.

3
docs/parser/lexer.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Lexer
The lexer is the code generated by the [flexer](./flexer.md) that is actually
responsible for lexing Enso source code. It chunks the character stream into a
(structured) token stream in order to make later processing faster, and to
@ -20,6 +21,7 @@ identify blocks
<!-- /MarkdownTOC -->
## Lexer Functionality
The lexer needs to provide the following functionality as part of the parser.
- It consumes the source lazily, character by character, and produces a
@ -28,6 +30,7 @@ The lexer needs to provide the following functionality as part of the parser.
token stream, represented by `Invalid` tokens.
## The Lexer AST
In contrast to the full parser [ast](./ast.md), the lexer operates on a
simplified AST that we call a 'structured token stream'. While most lexers
output a linear token stream, it is very important in Enso that we encode the

3
docs/parser/macro-resolution.md
View File

@ -7,6 +7,7 @@ order: 5
---
# Macro Resolution
Macro resolution is the process of taking the structured token stream from the
[lexer](./lexer.md), and resolving it into the [ast](./ast.md) through the
process of resolving macros. This process produces a chunked AST stream,
@ -20,6 +21,7 @@ including spacing-unaware elements.
<!-- /MarkdownTOC -->
## Functionality
The current functionality of the macro resolver is as follows:
- TBC
@ -36,6 +38,7 @@ The current overview of the macro resolution process can be found in the scala
> - Create a detailed design for how macro resolution should work.
## Errors During Macro Resolution
It is very important that, during macro resolution, the resolver produces
descriptive errors for error conditions in the macro resolver.

2
docs/parser/operator-resolution.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Operator Resolution
Operator resolution is the process of resolving applications of operators into
specific nodes on the AST.
@ -17,6 +18,7 @@ specific nodes on the AST.
<!-- /MarkdownTOC -->
## Resolution Algorithm
The operator resolution process uses a version of the classic
[shunting-yard algorithm](https://en.wikipedia.org/wiki/Shunting-yard_algorithm)
with modifications to support operator sections.

2
docs/parser/parser-driver.md
View File

@ -7,6 +7,7 @@ order: 8
---
# Parser Driver
The parser driver component is responsible for orchestrating the entire action
of the parser. It handles the following duties:
@ -22,6 +23,7 @@ of the parser. It handles the following duties:
<!-- /MarkdownTOC -->
## Driver Clients
The parser is going to be employed in two contexts, both running in-process:
1. In the IDE codebase as a rust dependency.

9
docs/parser/reader.md
View File

@ -7,6 +7,7 @@ order: 11
---
# Reading Source Code
The reader is responsible for abstracting the interface to reading a character
from a stream. This handles abstracting away the various encodings that the
project is going to use, as well as backing formats for the stream.
@ -15,12 +16,13 @@ project is going to use, as well as backing formats for the stream.
- [Reader Functionality](#reader-functionality)
- [Provided Readers](#provided-readers)
- [UTF-8 Reader](#utf-8-reader)
- [UTF-16 Reader](#utf-16-reader)
- [UTF-8 Reader](#utf-8-reader)
- [UTF-16 Reader](#utf-16-reader)
<!-- /MarkdownTOC -->
## Reader Functionality
The reader trait needs to have the following functionality:
- It must read its input _lazily_, not requiring the entire input to be in
@ -29,14 +31,17 @@ The reader trait needs to have the following functionality:
UTF-8, and hence abstract away the various underlying encodings.
## Provided Readers
The parser implementation currently provides the following reader utilities to
clients.
### UTF-8 Reader
Rust natively uses UTF-8 encoding for its strings. In order for the IDE to make
use of the parser, it must provide a simple rust-native reader.
### UTF-16 Reader
As the JVM as a platform makes use of UTF-16 for encoding its strings, we need
to provide a reader that will let JVM clients of the parser provide the source
code in a streaming fashion without needing to re-encode it prior to passing it

5
docs/parser/tech-analysis.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Parser Technology Analysis
As the Enso parser has some fairly unique requirements placed upon it, the
choice of implementation technology is of paramount importance. Choosing the
correct technology ensures that we can meet all of the requirements placed upon
@ -22,6 +23,7 @@ the parser.
<!-- /MarkdownTOC -->
## Technology Requirements for the Parser
As the parser has to work both for the Engine and for the IDE, it has a strange
set of requirements:
@ -34,6 +36,7 @@ set of requirements:
macOS, Linux and Windows.
## Issues With the Previous Implementation
The previous implementation of the parser was implemented in Scala, and had some
serious issues that have necessitated this rewrite:
@ -48,6 +51,7 @@ serious issues that have necessitated this rewrite:
textual operations under the hood, this proved to be a significant slowdown.
## Choosing Rust
Rust, then, is an obvious choice for the following reasons:
- It can be compiled _natively_ into the IDE binary, providing them with
@ -59,6 +63,7 @@ Rust, then, is an obvious choice for the following reasons:
interpreter to execute the parser safely in a non-native context.
### Downsides of Rust
This is not to say that choosing rust doesn't come with some compromises:
- It significantly complicates the CI pipeline for the engine, as we will have

3
docs/polyglot/README.md
View File

@ -7,13 +7,14 @@ order: 0
---
# Enso Polyglot Support
Enso supports robust polyglot interoperation with other programming languages
that are supported on its platform. This section of the design documentation
deals with
It is broken down into the following sections:
- [**Polyglot Bindings:**](./polyglot-bindings.md) A document providing an
- [**Polyglot Bindings:**](./polyglot-bindings.md) A document providing an
overview of the mechanisms provided to work with polyglot bindings in Enso.
- [**Typing Polyglot Bindings:**](./typing-polyglot-bindings.md) An exploration
of how we can provide a modicum of type safety for the polyglot bindings in

12
docs/polyglot/java.md
View File

@ -7,8 +7,9 @@ order: 3
---
# Polyglot Java
This document deals with the implementation of polyglot interoperation with
Java in the runtime. Please familiarise yourself with the general operation of
This document deals with the implementation of polyglot interoperation with Java
in the runtime. Please familiarise yourself with the general operation of
[polyglot bindings](./polyglot-bindings.md).
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -20,6 +21,7 @@ Java in the runtime. Please familiarise yourself with the general operation of
<!-- /MarkdownTOC -->
## Class Lookup
In order for the Enso runtime to effectively find Java objects for working with
in a polyglot fashion, it will look in the `polyglot/java` subdirectory of an
Enso project. This directory has the following requirements placed on it.
@ -37,12 +39,13 @@ Enso project. This directory has the following requirements placed on it.
> maybe even compiling Java code.
## Polyglot Library System
The dynamic polyglot system is a dynamic runtime lookup for Java objects,
allowing Enso code to work with them through a runtime reflection-style
mechanism. It is comprised of the following components:
- `Java.lookup_class : Class.Path -> Maybe Class`: A function that lets
users look up a class by a given name on the runtime classpath.
- `Java.lookup_class : Class.Path -> Maybe Class`: A function that lets users
look up a class by a given name on the runtime classpath.
- `Polyglot.instantiate : Class -> Object`: A function that lets users
instantiate a class into an object.
- A whole host of functions on the polyglot type that let you dynamically work
@ -63,6 +66,7 @@ main =
> - Expand on the detail when there is time.
## Polyglot Syntax System
The static system, however, lets us do much better in terms of user experience.
Instead of having to dynamically look things up at runtime, we can instead do
the following:

14
docs/polyglot/polyglot-bindings.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Polyglot Bindings
This document deals with the specification and design for the polyglot interop
system provided in the Enso runtime. This system allows users to connect Enso to
other supported programming languages, to both provide access to a wealth of
@ -36,6 +37,7 @@ main mechanisms:
<!-- /MarkdownTOC -->
## Impedance Mismatch
Polyglot interoperation in Enso has a significant impedance mismatch. In
essence, this means that there is a mismatch between Enso's language semantics
and the semantics of the foreign languages that are being worked with.
@ -49,6 +51,7 @@ represented by Enso.
> - Expand on the impedance mismatch and how it leads to the defined semantics.
## The Polyglot FFI
The low-level polyglot FFI mechanism refers to a way to use polyglot objects
directly in Enso code. This can be used to underlie a library implementaion in
Enso, or to interoperate with code running in other languages.
@ -57,6 +60,7 @@ The mechanism provides users with the facilities to import bindings from other
languages and call them via a generic mechanism.
### Importing Polyglot Bindings
When importing a polyglot binding into scope in an Enso file, this introduces a
_polyglot object_ into scope. This object will have appropriate fields and/or
methods defined on it, as described by the foreign language implementation.
@ -67,6 +71,7 @@ methods defined on it, as described by the foreign language implementation.
> clearer.
### Using Polyglot Bindings
With a polyglot object in scope, the user is free to call methods on it
directly. These polyglot objects are inherently dynamically typed, meaning that
any operation may _fail_ at runtime.
@ -93,6 +98,7 @@ two things which have no real equivalent in the Enso type system.
> - Determine how to make the inherent 'failability' of polyglot objects safer.
### Importing Polyglot Bindings (Syntax)
Polyglot bindings can be imported using a polyglot import directive. This is
constructed as follows:
@ -112,6 +118,7 @@ polyglot c import struct NetworkPacket as NetworkPacketC
```
### Using Polyglot Bindings (Syntax)
A polyglot binding is a polyglot object that has methods and/or fields defined
on it. Due to an impedance mismatch between languages, Enso implements a
variadic syntax for calling these polyglot bindings using vectors.
@ -142,6 +149,7 @@ main =
```
### Finding Polyglot Bindings
Polyglot objects for various languages are found in the `polyglot` subdirectory
of an Enso project. This folder is subdivided into directories based on the
polyglot language. The name of each subdirectory must match the language
@ -152,18 +160,20 @@ implementation for that particular language needing to specify it. Please see
the language-specific documentation for details.
## Embedded Syntax
The term "Embedded Syntax" is our terminology for the ability to use foreign
language syntaxes from directly inside `.enso` files. This system builds upon
the more generic mechanisms used by the [polyglot FFI](#the-polyglot-ffi) to
provide a truly seamless user experience.
### Embedded Syntax Usage (Syntax)
A polyglot block is introduced as follows:
- The `polyglot` keyword starts a block.
- This must be followed by a language identifier (e.g. `java`).
- After the language identifier, the remaining syntax behaves like it is an
Enso function definition until the `=`.
- After the language identifier, the remaining syntax behaves like it is an Enso
function definition until the `=`.
- After the `=`, the user may write their foreign code.
```ruby

11
docs/polyglot/typing-polyglot-bindings.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Typing the Polyglot Bindings
The polyglot bindings inherently provide a problem for the Enso type system.
When many of the languages with which we can interoperate are highly dynamic and
flexible, or have significant mismatches between their type system and Enso's,
@ -18,11 +19,12 @@ boundary.
- [Enso Values](#enso-values)
- [Polyglot Values](#polyglot-values)
- [Dynamic](#dynamic)
- [The Enso Boundary](#the-enso-boundary)
- [The Enso Boundary](#the-enso-boundary)
<!-- /MarkdownTOC -->
## Enso Values
The underlying nature of our runtime allows us to pass Enso values across the
polyglot boundary while ensuring that they aren't modified. This means that the
typing information known about a value `v` _before_ it is passed to a polyglot
@ -41,10 +43,11 @@ case, the value needs to be treated as `Dynamic` after its use.
> - Think much more on this.
## Polyglot Values
In the presence of a polyglot value, however, there is very little that we can
determine about a value with which we are working. This means that we need to
have a principled way to assert properties on a polyglot object that can then
be reflected in the Enso type system. This mechanism needs to deal with:
have a principled way to assert properties on a polyglot object that can then be
reflected in the Enso type system. This mechanism needs to deal with:
- Concurrent access to polyglot objects.
- Mutation and modification of polyglot objects.
@ -56,6 +59,7 @@ be reflected in the Enso type system. This mechanism needs to deal with:
> could take to it.
## Dynamic
As Enso can seamlessly interoperate with other programming languages, we need a
principled way of handling dynamic types that we don't really know anything
about. This mechanism needs:
@ -90,6 +94,7 @@ obj.model =
> could self-modify underneath us.
### The Enso Boundary
Fortunately, we can at least avoid foreign languages modifying memory owned by
the Enso interpreter. As part of the interop library, Graal lets us mark memory
as read-only. This means that the majority of data passed out (from a functional

24
docs/rfcs/0001-rfc-template.md
View File

@ -5,26 +5,28 @@ category: rfcs
tags: [rfcs, specification]
order: 1 # Should match the RFC Number
---
---
> **Feature Name:** A unique name for the feature
> **Start Date:** Today's date (YYYY-MM-DD)
> **Change Type:** Breaking / Non-Breaking (delete as appropriate)
> **RFC Dependencies:** List any RFCs that this one depends on.
> **RFC PR:** Leave Empty
> **Enso Issue:** Leave Empty
> **Implemented:** Leave blank, this will be filled with the first version of
the relevant tool where it is implemented.
> - **Feature Name:** A unique name for the feature
> - **Start Date:** Today's date (YYYY-MM-DD)
> - **Change Type:** Breaking / Non-Breaking (delete as appropriate)
> - **RFC Dependencies:** List any RFCs that this one depends on.
> - **RFC PR:** Leave Empty
> - **Enso Issue:** Leave Empty
> - **Implemented:** Leave blank, this will be filled with the first version of
> the relevant tool where it is implemented.
# Summary
A one-paragraph, high-level summary of the proposal.
# Motivation
Why should we make this change? What use-cases does it support? What benefits
does it bring to the ecosystem? Explain why the status quo is insufficient or
not ideal.
# Guide-Level Explanation
Explain the proposal as if teaching the feature(s) to a newcomer to Enso. This
should usually include:
@ -44,6 +46,7 @@ impact. For policy-level RFCs, this section should provide an example-driven
introduction to the policy, and explain its impact in concrete terms.
# Specification-Level Explanation
This is the technical portion of the RFC and should be written as a
specification of the new feature(s). It should provide a syntax-agnostic
description of the planned feature, and include sufficient detail to address the
@ -62,10 +65,12 @@ include where relevant:
- The types and semantics of any new library interfaces.
# Drawbacks
A description of why we _should not_ do this. Write this section as if you are
picking apart your proposal.
# Rationale and Alternatives
A few paragraphs addressing the rationale for why this design is the best
possible design for this feature in its design space. It should address how the
proposed design resolves the motivating factors.
@ -76,6 +81,7 @@ for not choosing them.
A paragraph or two addressing the impact of not including this feature.
# Unresolved Questions
This section should address any unresolved questions you have with the RFC at
the current time. Some examples include:

28
docs/rfcs/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# The Enso RFC Process
Like any Open Source language, Enso welcomes interaction from the community, as
we believe that only through the community's suggestions and feedback can we
make Enso the best it can be.
@ -14,8 +15,8 @@ make Enso the best it can be.
To this end, this repository defines a process to be followed for those people
that want to make 'substantial' changes to the Enso language, the Enso compiler
and runtime, or even this RFC process itself. What we define as a 'substantial'
change is in flux based on community norms and in relation to the portion of
the ecosystem being changed.
change is in flux based on community norms and in relation to the portion of the
ecosystem being changed.
Substantial changes may thus include the following:
@ -54,15 +55,17 @@ you have signed the [CLA](../CONTRIBUTING.md#the-contributor-license-agreement).
<!-- /MarkdownTOC -->
## Creating an RFC
While informal, we do aim to provide a basic outline for the process of creating
an RFC. The intention of this process is that RFCs arrive in a state with enough
detail to spark discussion, and then evolve through discussion with all
interested parties.
### Before Creating an RFC
Keep in mind that a hastily-created RFC can hurt discussion and its chances of
being accepted, even if the idea proposed is excellent! A half-baked proposal,
a previously-rejected idea, or a feature that doesn't align with our vision for
being accepted, even if the idea proposed is excellent! A half-baked proposal, a
previously-rejected idea, or a feature that doesn't align with our vision for
Enso could be rejected swiftly, making the unprepared contributor feel
demotivated. Doing a little bit of preparatory legwork can make the RFC process
that much more smooth.
@ -75,6 +78,7 @@ If the idea receives encouraging feedback from the core team, as well as the
community, it is an indication that the RFC is worth pursuing.
### The RFC Process
To summarise the end goal of the RFC process, the aim is to merge the proposal
(as a markdown file) into the RFC Repository. At this point, the RFC becomes
'Active', and may then be implemented.
@ -94,11 +98,11 @@ The process for creating an RFC can be outlined as follows:
of the key points for considering RFCs is how they fit with the vision for
Enso as a whole.
4. Submit a [Pull Request](https://github.com/enso-org/enso/pulls). Please give
the PR a descriptive title (`RFC: My Feature`). The Pull Request will be
open for feedback and discussion from the community and core team, and the
author should be open to revising it in response to this feedback. The RFC
will be assigned a shepherd from the core team who will be responsible for
managing the RFC alongside the author.
the PR a descriptive title (`RFC: My Feature`). The Pull Request will be open
for feedback and discussion from the community and core team, and the author
should be open to revising it in response to this feedback. The RFC will be
assigned a shepherd from the core team who will be responsible for managing
the RFC alongside the author.
5. Evolve the RFC. Build consensus around the feature through your revisions.
Feel free to reach out to the shepherd assigned to your proposal for
discussion, as well as help identifying the key stakeholders and obstacles.
@ -117,12 +121,14 @@ The process for creating an RFC can be outlined as follows:
Postponed, where the PR is tagged and closed, or Rejected and closed.
### Reviewing RFCs
While the RFC pull request is under discussion, the shepherd may organise
meetings with the author and stakeholders to discuss the issue(s) in more
detail. A summary from this meeting will be compiled by the shepherd and posted
back to the pull request.
## The RFC Life-Cycle
RFCs proceed through their life-cycle in this repository as follows:
1. The Pull Request is submitted. The repository manager assigns appropriate
@ -136,6 +142,7 @@ RFCs proceed through their life-cycle in this repository as follows:
5. Once merged, implementation can be commenced and pull requested.
### Active RFCs
An RFC being 'Active' should not be viewed as a rubber stamp, and in particular
it does _not_ mean that the feature will be ultimately merged into Enso. It does
mean that, in principle, stakeholders are in agreement on the feature and are
@ -157,6 +164,7 @@ substantial changes are required, they should take place as a new RFC, linked to
and from the old.
### RFC Implementation
Some accepted RFCs are vital features that need to be implemented as soon as
possible, while others can wait until a so-inclined developer comes along. Every
accepted RFC is assigned a tracking issue in the
@ -172,6 +180,7 @@ cannot determine if someone else is already working on it, feel free to ask
(e.g. by leaving a comment on the associated issue).
### RFC Postponement
Some RFC pull requests are tagged with the 'Postponed' label when they are
closed (as part of the rejection process). This label implies that we neither
want to think about evaluating the proposal or implementing it until some time
@ -182,6 +191,7 @@ is usually given because the time is not right to consider or implement such a
feature for now.
## Credits
This repository and process takes significant inspiration from the
[Rust RFCs](https://github.com/rust-lang/rfcs) process, and the
[GHC Proposals](https://github.com/ghc-proposals/ghc-proposals) process, so

1
docs/runtime/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso Runtime
The Enso runtime refers to both the compiler and the optimising JIT runtime for
Enso. While this might seem like a strange choice, it makes sense internally as
the components are integrated to a level not seen in most languages. It

71
docs/runtime/builtin-base-methods.md
View File

@ -7,6 +7,7 @@ order: 7
---
# Builtin Base Methods
While we strive to implement most of the Enso standard library in Enso itself,
it is necessary for certain methods, particularly ones involving operations out
of reach for the standard language semantics, or primitive system calls, to be
@ -26,13 +27,14 @@ generation of such methods' boilerplate.
<!-- /MarkdownTOC -->
## DSL Overview
The Enso Base DSL exposes certain annotations that allow to generate
boilerplate transforming standard Truffle nodes into a shape that can easily
be exported to the Base library. This DSL facilitates automatic parsing
and typechecking of method arguments and wrapping the node in an actual
`Function` object.
The Enso Base DSL exposes certain annotations that allow to generate boilerplate
transforming standard Truffle nodes into a shape that can easily be exported to
the Base library. This DSL facilitates automatic parsing and typechecking of
method arguments and wrapping the node in an actual `Function` object.
## `@BuiltinMethod` Annotation
The `@BuiltinMethod` annotation is applied to a Truffle `Node` subclass that
should be treated as a builtin method. It takes the following arguments:
@ -47,51 +49,54 @@ should be treated as a builtin method. It takes the following arguments:
in a non-tail position, this parameter should be left defaulted.
The annotation will generate a `RootNode` subclass in the same package as the
declaring node, with a name basing on the declaring node's name. The name of
the generated node is the name of the original node, with the `Node` suffix
stripped and the `MethodGen` suffix appended. E.g. `AndOperatorNode` will
generate a `AndOperatorMethodGen` root node.
declaring node, with a name basing on the declaring node's name. The name of the
generated node is the name of the original node, with the `Node` suffix stripped
and the `MethodGen` suffix appended. E.g. `AndOperatorNode` will generate a
`AndOperatorMethodGen` root node.
## Structural Requirements
The DSL places certain structural requirements on the declaring node.
1. **Construction:** The node must be constructible using either
a paremeterless, static `build` method or a parameterless constructor.
If a suitable `build` method is defined, it will be preferred over the
1. **Construction:** The node must be constructible using either a
paremeterless, static `build` method or a parameterless constructor. If a
suitable `build` method is defined, it will be preferred over the
constructor.
2. **Execution:** The node must declare a single, accessible (i.e. at least
package-private), non-void method named `execute`.
## `Execute` Method Semantics
The `execute` method defined by the declaring node has the following semantics.
1. Return type. If the return type is `Stateful`, the method is assumed to
modify the monadic state. For any other return type, the method cannot
modify the monadic state.
2. Arguments. The method may take any number of arguments, though at least
an argument named `self` must be present. The arguments have following
modify the monadic state. For any other return type, the method cannot modify
the monadic state.
2. Arguments. The method may take any number of arguments, though at least an
argument named `self` must be present. The arguments have following
semantics:
1. An `Object` argument may be annotated with `@MonadicState`. This tells
the DSL to substitute it for the current value of monadic state. Note
that this value is read-only. In order to modify state, the method
must return a `Stateful`.
1. An `Object` argument may be annotated with `@MonadicState`. This tells the
DSL to substitute it for the current value of monadic state. Note that
this value is read-only. In order to modify state, the method must return
a `Stateful`.
2. Any `VirtualFrame` argument will be passed the current execution frame.
3. Any `CallerInfo` argument will be passed the current `CallerInfo` and
mark the method as requiring the caller info at runtime.
3. Any `CallerInfo` argument will be passed the current `CallerInfo` and mark
the method as requiring the caller info at runtime.
4. All other arguments are treated as positional arguments to the method.
Note that according to the language specification, any method must take
an argument named `this`. Because of a naming conflict with Java, this
argument is called `_this` in the DSL. Please note that it is mandatory
to take an argument called `_this`, even if it is unused.
5. The positional arguments can be of any typed specified by the runtime
type system (see `org.enso.interpreter.runtime.Types`) or `Object`.
Arguments typed as `Thunk` mark the argument as suspended in the
generated function header.
Note that according to the language specification, any method must take an
argument named `this`. Because of a naming conflict with Java, this
argument is called `_this` in the DSL. Please note that it is mandatory to
take an argument called `_this`, even if it is unused.
5. The positional arguments can be of any typed specified by the runtime type
system (see `org.enso.interpreter.runtime.Types`) or `Object`. Arguments
typed as `Thunk` mark the argument as suspended in the generated function
header.
## Generated Node
The DSL generates a single root node per annotated class. This node handles
reading the arguments from the execution frame and typechecking them, before
delegating to the declaring node.
The generated node also defines a static `makeFunction(Language)` method,
wrapping the node in a runtime function object.
wrapping the node in a runtime function object.

86
docs/runtime/caching.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Caching
It is not uncommon for users in data-analysis jobs to work with data on the
order of _gigabytes_ or even _terabytes_. As fast as computers have become, and
as efficient as programming languages can be, you still don't want to compute on
@ -56,6 +57,7 @@ than having to recompute the entire program.
<!-- /MarkdownTOC -->
## Cache Candidates
The key use of the Enso value cache is to support the interactive editing of
user code. This means that it caches all bindings within the scope of a given
function, including the function arguments. This means that, as users edit their
@ -77,25 +79,27 @@ this function (in this case `c` and `d`), as well as the inputs to the function
All intermediate results and inputs are considered as candidates, though as the
cache design evolves, the _selected_ candidates may be refined. Ultimately we
want to cache and reuse as much as possible to minimize the computation
costs. At the same time, we want to limit the memory consumed by the cache.
want to cache and reuse as much as possible to minimize the computation costs.
At the same time, we want to limit the memory consumed by the cache.
### Initial Cache Candidates
The initial version of the cache only stores the right-hand-sides of binding
expressions. This is for two main reasons:
- Firstly, this allows us to ensure that the cache does not cause the JVM to
go out of memory between executions, allowing us to avoid the implementation
of a [memory-bounded cache](#memory-management) for now.
- Firstly, this allows us to ensure that the cache does not cause the JVM to go
out of memory between executions, allowing us to avoid the implementation of a
[memory-bounded cache](#memory-management) for now.
- It also simplifies the initial implementation of weighting program components.
### Further Development of Cache Candidates
The next step for the cache is to expand the portions of the introspected scope
that we cache. In general, this means the caching of intermediate expressions.
However, once we do this, we can no longer guarantee that we do not push the
JVM out of memory between two program executions. This is best demonstrated
by example.
However, once we do this, we can no longer guarantee that we do not push the JVM
out of memory between two program executions. This is best demonstrated by
example.
```
a = (computeHugeObject b).size
@ -107,6 +111,7 @@ the intermediate result of the discarded `computeHugeObject b` expression, we
need some way of tracking the sizes of individual cache entries.
## Partial-Evaluation and Side-Effects
The more theoretically-minded people among those reading this document may
instantly realise that there is a _problem_ with this approach. In the presence
of caching, it becomes _entirely_ unpredictable as to when side effects are
@ -121,6 +126,7 @@ In general, it is clear that many kinds of side effect have _problems_ in the
presence of caching and partial-evaluation.
### Side Effects in the Initial Version
Many of the mechanisms required to deal with this kind of issue properly are
complex and require deep type-level support in the compiler. To that end, the
initial version of the interpreter is going to pretend that the problem doesn't
@ -134,6 +140,7 @@ This can and _will_ recompute side-effecting computations indiscriminately, but
we cannot initially do much better.
#### A Stopgap
While the compiler won't have the machinery in place to properly track
information about side-effects, we can implement a stop-gap solution that at
least allows the GUI to allow users to make the decision about whether or not to
@ -148,6 +155,7 @@ used for functions with arguments marked as `Suspended`, and works as follows:
opt in to automatic re-evaluation of these values.
### In The Future
As the compiler evolves, however, we can do better than this. In particular, we
can employ type-system information to determine which functions are
side-effecting (in absence of annotations), and to class some kinds of said
@ -171,10 +179,12 @@ brief sketch of how this might work:
> capabilities that will be useful in doing so.
## Cache Eviction Strategy
The cache eviction strategy refers to the method by which we determine which
entries in the cache are invalidated (if any) after a given change to the code.
### Initial Eviction Strategy
In the initial version of the caching mechanism, the eviction strategies are
intended to be fairly simplistic and conservative to ensure correctness.
@ -192,26 +202,28 @@ changed:
the dynamic symbol are evicted from the cache.
### Future Eviction Strategies
In the future, however, the increasing sophistication of the front-end compiler
for Enso will allow us to do better than this by accounting for more granular
information in the eviction decisions.
Cache eviction takes into account the following aspects:
* **Visualization:** In the first place, we should care about nodes that have
- **Visualization:** In the first place, we should care about nodes that have
visualization attached in the IDE.
* **Priority:** The runtime can assign a score to a node, meaning how valuable
- **Priority:** The runtime can assign a score to a node, meaning how valuable
this node is. Less valuable nodes should be evicted first.
* **Computation time:** The runtime can calculate the time that node took to
- **Computation time:** The runtime can calculate the time that node took to
compute. Less computationally intensive nodes should be evicted first. Memory
limit. The cache should not exceed the specified memory limit.
> The actionables for this section are:
>
> - Evolve the cache eviction strategy by employing more granular information
> as the compiler evolves to provide it.
> - Evolve the cache eviction strategy by employing more granular information as
> the compiler evolves to provide it.
## Dataflow Analysis
Dataflow analysis is the process by which the compiler discovers the
relationships between program expressions. The output of the process is a data
dependency graph that can be queried for an expression, and returns the set of
@ -232,6 +244,7 @@ Each expression in the compiler IR is annotated with both the set of expressions
that depend on it, and the set of expressions that it depends on.
### Identifying Expressions
Expressions are identified, for the purposes of dataflow analysis, by unique
identifiers on every IR node. The dataflow analysis process creates a dependency
graph between these identifiers.
@ -242,11 +255,13 @@ identifiers is left to the runtime and is not the responsibility of the dataflow
analysis pass.
### Specifying Dataflow
Dataflow analysis takes place on the core set of language constructs, defined
as those that extend `IRKind.Primitive`. Their dataflow is specified as follows,
Dataflow analysis takes place on the core set of language constructs, defined as
those that extend `IRKind.Primitive`. Their dataflow is specified as follows,
with arrows representing edges in the graph.
#### Atom
An atom is dependent on the definitions of its arguments, particularly with
regard to any defaults.
@ -255,6 +270,7 @@ atom <- arguments
```
#### Method
A method is dependent on the definition of its body. Methods at the point that
dataflow analysis runs are 'bare' methods, meaning that they are defined as
functions.
@ -264,6 +280,7 @@ method <- body
```
#### Block
The value of a block is dependent purely on the value of its return expression.
The return expression may depend on other values.
@ -272,6 +289,7 @@ block <- returnValue
```
#### Binding
The value of a binding is dependent both on the name of the binding and the
expression being assigned in the binding.
@ -281,6 +299,7 @@ binding <- expression
```
#### Lambda
The value of a lambda is dependent on the return value from its body, as well as
the definitions of any defaults for its arguments.
@ -290,6 +309,7 @@ lambda <- argumentDefaults
```
#### Definition Argument
The value of a function definition argument is dependent purely on the value of
its default, if that default is present.
@ -298,6 +318,7 @@ defArgument <- defaultExpression
```
#### Prefix Application
The value of a prefix application is dependent on the values of both the
function expression being called, and the arguments.
@ -307,6 +328,7 @@ prefix <- arguments
```
#### Call Argument
The value of a call argument is dependent both on the value that it's wrapping,
as well as the name it has, if it exists.
@ -316,6 +338,7 @@ callArgument <- argumentName
```
#### Forced Term
A forced term is purely dependent on the value of the term being forced (the
`target`).
@ -324,6 +347,7 @@ force <- target
```
#### Typeset Members
A typeset member is dependent on the definition of its label, as well as the
possibly present definitions of its type and value.
@ -334,6 +358,7 @@ typesetMember <- memberValue
```
#### Typing Operators
All typing operators in Enso (`IR.Type`) are dependent on their constituent
parts:
@ -342,6 +367,7 @@ typingExpr <- expressionChildren
```
#### Name
An occurrence of a name is dependent on the definition site of that name. This
means that it is broken down into two options:
@ -351,6 +377,7 @@ means that it is broken down into two options:
```
name <- staticUseSite
```
2. **Dynamic Dependency:** The definition site for a given usage can only be
determined to be a symbol resolved dynamically.
@ -363,8 +390,9 @@ means that it is broken down into two options:
used the changed definition in particular.
#### Case Expressions
The value of a case expression depends on the value of its scrutinee, as well
as the definitions of its branches.
The value of a case expression depends on the value of its scrutinee, as well as
the definitions of its branches.
```
case <- scrutinee
@ -373,6 +401,7 @@ case <- fallback
```
#### Case Branches
The value of a case branch depends on both the pattern expression and the result
expression.
@ -382,6 +411,7 @@ caseBranch <- expression
```
#### Comments
The value of a comment is purely dependent on the value of the commented entity.
```
@ -389,15 +419,17 @@ comment <- commented
```
## Cache Backend
The cache is implemented as key-value storage with an eviction function.
The cache stores the right-hand side expressions of the bindings in the
key-value storage. The storage can be as simple as a Hash Map with values
wrapped into the Soft References as a fallback strategy of clearing the
cache. The eviction function purges invalidated expressions from previous
program execution.
wrapped into the Soft References as a fallback strategy of clearing the cache.
The eviction function purges invalidated expressions from previous program
execution.
### Further development
Cache intermediate results of expressions to reduce the cost of new computations
and extend the eviction strategy to clear the cache based on memory consumption.
@ -414,6 +446,7 @@ hit ratio.
> information of the stored values
## Memory Bounded Caches
Memory management refers to a way of controlling the size of the cache, avoiding
the Out Of Memory errors.
@ -426,6 +459,7 @@ In general, to control the memory size on JVM, anything besides limiting the
total amount involves some tradeoffs.
### Soft References
Soft References is a way to mark the cache entries available for garbage
collection whenever JVM runs a GC. In practice, it can cause long GC times and
reduced overall performance. This strategy is generally considered as a last
@ -442,35 +476,42 @@ the other hand, serialization itself can provide the size of the object, which
is enough to implement the eviction policy, even without running the _isolates_.
### Serialization
One approach to get the size of the value stored in the cache is
_serialization_. The downside is the computational overhead of transforming the
object into a byte sequence.
### Instrumentation
Another way of getting the object's size is to use JVM instrumentation. This
approach requires running JVM with _javaagent_ attached, which can complicate
the deployment and have indirect performance penalties.
### Manual Memory Management
This method implies tracking the size of the values by hand. It can be done for
the values of Enso language, knowing the size of its primitive types. For
interacting with other languages, Enso can provide an interface that developers
should implement to have a better experience with the cache.
### Comparison of Memory Management Approaches
Below are some key takeaways after experimenting with the _instrumentation_ and
_serialization_ approaches.
#### Enso Runtime Benchmark
The existing runtime benchmark was executed with the java agent attached to
measure the impact of instrumentation.
#### Serialization Benchmark
[FST](https://github.com/RuedigerMoeller/fast-serialization) library was used
for the serialization benchmark. It doesn't require an explicit scheme and
relies on Java `Serializable` interface.
#### Instrumentation Benchmark
Java
[`Instrumentation#getObjectSize()](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html)
can only provide a _shallow_ memory of the object. It does not follow the
@ -481,11 +522,12 @@ Benchmark used the `MemoryUtil#deepMemoryUsageOf` function of
reflection to follow the references and access the private fields of the object.
#### Benchmark Results
Benchmarks measured Java array, `java.util.LinkedList`, and a custom
implementation of lined list `ConsList`, an object that maintains references for
its head and tail.
``` java
```java
public static class ConsList<A> {
private A head;
@ -497,7 +539,7 @@ its head and tail.
Function execution time measured in milliseconds per operation (lower is
better).
``` text
```text
Benchmark (size) Mode Cnt Score Error Units
FstBenchmark.serializeArray 1000 avgt 5 21.862 ± 0.503 us/op
FstBenchmark.serializeConsList 1000 avgt 5 151.791 ± 45.200 us/op

8
docs/runtime/demand-analysis.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Demand Analysis
Demand analysis is the process of deciding when to compute the value of a
suspended computation in a language which supports suspended computation.
@ -19,10 +20,11 @@ suspended computation in a language which supports suspended computation.
<!-- /MarkdownTOC -->
## Determining When To Force
As Enso supports dynamic dispatch for methods, we cannot always (even in the
presence of a typechecker), statically determine whether or not the function
presence of a typechecker), statically determine whether or not the function
that will eventually be called at any given call site. This means that we have
to come up with a more efficient way to handle suspended arguments.
to come up with a more efficient way to handle suspended arguments.
This is done by making the _function application_ responsible for determining
when a value passed to it should be evaluated. It works as follows:
@ -36,6 +38,7 @@ when a value passed to it should be evaluated. It works as follows:
be explicitly forced at their use sites in the function body.
## Avoiding Pathological Force-Thunk Chains
The above approach does, when done naively, result in a severe performance
pathology when passing suspended arguments into functions also expecting
suspended arguments. Without intervention, the suspended argument gets wrapped
@ -49,6 +52,7 @@ In order to avoid this issue, we do the following:
2. All other uses of that argument are forced.
## The Demand Analysis Algorithm
The algorithm for performing demand analysis on Enso code is as follows.
```

39
docs/runtime/execution-server-flow.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Execution Server Flow
This document describes the API and workflow of the internal execution server.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -14,45 +15,52 @@ This document describes the API and workflow of the internal execution server.
- [Opening a Connection](#opening-a-connection)
- [API](#api)
- [Internal Architecture](#internal-architecture)
- [Job Queue](#job-queue)
- [Job Types](#job-types)
- [Scheduling Rules](#scheduling-rules)
- [API Methods to Jobs Translation](#api-methods-to-jobs-translation)
- [Job Queue](#job-queue)
- [Job Types](#job-types)
- [Scheduling Rules](#scheduling-rules)
- [API Methods to Jobs Translation](#api-methods-to-jobs-translation)
<!-- /MarkdownTOC -->
## Opening a Connection
> The actionables for this section are:
>
>
> 1. describe the `org.graalvm.polyglot.Context.Builder.serverTransport`
> workflow of connecting to the server.
## API
> The actionables for this section are:
>
>
> 1. Document the server's API.
## Internal Architecture
This section describes certain implementation details of the execution server,
allowing it to perform its operations safely and interactively.
### Job Queue
The execution server uses a job queue containing requests to be performed.
An API method may queue multiple jobs.
The execution server uses a job queue containing requests to be performed. An
API method may queue multiple jobs.
### Job Types
There are a number of job types used internally for handling different
scenarios:
#### `EnsureCompiled`
Takes a set of module names that must be compiled and ensures that the
corresponding modules are compiled in the newest version.
It also performs cache invalidations on changes since the last batch.
Caches should be invalidated for all contexts affected by this recompilation.
This operation is idempotent and should be run before any `Execute` action.
This operation is not currently interruptible, but may become so in the future.
corresponding modules are compiled in the newest version. It also performs cache
invalidations on changes since the last batch. Caches should be invalidated for
all contexts affected by this recompilation. This operation is idempotent and
should be run before any `Execute` action. This operation is not currently
interruptible, but may become so in the future.
#### `Execute`
Takes a context ID and an optional set of expression IDs that should be
executed. and executes the Enso code corresponding to the context's stack.
Updates caches and sends updates to the users.
@ -73,6 +81,7 @@ This operation is interruptible through `Thread.interrupt()`.
6. The order of `EnsureCompiled` jobs can be freely changed by the compiler.
### API Methods to Jobs Translation
The following describes handling of API messages through job queue
modifications.
@ -92,8 +101,8 @@ modifications.
3. Visualization modifications:
1. Synchronously perform all state updates.
2. Respond to the user.
3. Enqueue `EnsureCompiled` and `Execute` for the affected context.
Set the minimal set of expressions required to compute to
3. Enqueue `EnsureCompiled` and `Execute` for the affected context. Set the
minimal set of expressions required to compute to
`Set(visualizedExpression)` in the `Execute` command.
4. Create/Destroy context:
1. Abort any jobs concerning the affected context.

56
docs/runtime/function-call-flow.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Function Calling Flow
With Enso being a functional language, it is crucial for function calls to be
efficient and easily inlined. At the same time, the logic of calling functions
is involved – there's a plethora of features this system needs to support,
@ -31,6 +32,7 @@ framework.
<!-- /MarkdownTOC -->
## Tail Call Optimization
It is very important for Enso to perform TCO well, since there are no intrinsic
looping constructs in the language.
@ -77,6 +79,7 @@ while true
This logic is encapsulated in the various subclasses of `CallOptimiserNode`.
## Named Application Arguments
Enso allows applying function arguments by name, e.g.
```ruby
@ -85,20 +88,19 @@ z = foo exp=10 base=3
```
While certainly useful for the user, it requires some non-trivial facilitation
in the interpreter. An easy solution, would be to simply pass arguments in
a dictionary-like structure (e.g. a HashMap), but that has unacceptable
in the interpreter. An easy solution, would be to simply pass arguments in a
dictionary-like structure (e.g. a HashMap), but that has unacceptable
performance characteristics.
Therefore, the approach of choice is to compute and _cache_ a translation
table – a recipe for reordering arguments for a given function at a given
call site.
Therefore, the approach of choice is to compute and _cache_ a translation table
– a recipe for reordering arguments for a given function at a given call site.
Based on the knowledge of the call-site schema (i.e. a description like
"3 arguments are provided, the first is named foo, the second is named bar and
the third is applied positionally") and the definition-site schema ("the
function has 3 parameters, named [baz, foo, bar]"), a mapping ("move the first
argument to the second position, the second becomes third and the third becomes
the first") is computed.
Based on the knowledge of the call-site schema (i.e. a description like "3
arguments are provided, the first is named foo, the second is named bar and the
third is applied positionally") and the definition-site schema ("the function
has 3 parameters, named [baz, foo, bar]"), a mapping ("move the first argument
to the second position, the second becomes third and the third becomes the
first") is computed.
This mapping is then memoized and used to efficiently reorder arguments on each
execution, without the need to employ any more involved data structures. A
@ -109,6 +111,7 @@ paths in the cache, as well as cache misses for highly polymorphic call sites.
This logic is encapsulated in the `ArgumentSorterNode`.
## Definition-Site Argument Suspension
Enso allows functions to define certain arguments as `Suspended`, so that when
these are passed to a function, the corresponding expressions are not evaluated
at the call site, but are instead passed to the function as closures for
@ -120,6 +123,7 @@ evaluated at call-site when the function signature defines them as eager.
Argument execution is happening inside the `ArgumentSorterNode`.
## Currying and Eta-Expansion
Functions can also be applied partially (i.e. with less arguments than required
by the signature, in which case the result is a function with certain arguments
fixed) or over-saturated (i.e. if a function returns another function, in which
@ -129,6 +133,7 @@ expression).
This logic is handled inside the `CurryNode`.
## Dynamic Dispatch
Functions can be dispatched dynamically, meaning a name can denote different
functions, based on the (runtime) type of the first argument.
@ -136,6 +141,7 @@ This logic is fairly straightforward. It is triggered from `InvokeCallableNode`
and performed (with caching) in the `MethodResolverNode`.
## Defaulted Arguments and Application
As we want to provide a consistent semantics in the language (especially with
the use of multi-argument lambdas internally), there is one specific situation
that arises when handling applications with default arguments. As we expect a
@ -148,23 +154,25 @@ function is a function, and if it is fully saturated with defaults it will call
it.
## Megamorphic Call Sites
In certain situations, such as call sites calling a lot of different functions
without splitting or intensive use of the same instance of the interop library,
caching intermediate information about the call-site becomes impossible.
For such cases, we use nodes with names starting with `Indirect` to perform
the required operation. These nodes do not perform any caching and instead use
the most general version of the logic possible for every call. These nodes also
caching intermediate information about the call-site becomes impossible. For
such cases, we use nodes with names starting with `Indirect` to perform the
required operation. These nodes do not perform any caching and instead use the
most general version of the logic possible for every call. These nodes also
support the `uncached` mode of operation, which means the node itself may be
used without adopting them, using a global singleton instance instead.
As such the indirect nodes are very slow and should only be used after it is
discovered that the call site cannot afford more specific operations anymore.
The `uncached` version of these nodes is even slower, as no inlining can happen
on sites using them. This should be used with care, only when large call
overhead is acceptable (e.g. rarely-used instances of the interop library).
used without adopting them, using a global singleton instance instead. As such
the indirect nodes are very slow and should only be used after it is discovered
that the call site cannot afford more specific operations anymore. The
`uncached` version of these nodes is even slower, as no inlining can happen on
sites using them. This should be used with care, only when large call overhead
is acceptable (e.g. rarely-used instances of the interop library).
## Flow Diagram
The following diagram summarizes all the nodes participating in a function
call. The entry points to this system are `ApplicationNode` (for in-language
function calls) and `InteropLibrary<Function>` (for polyglot function calls).
The following diagram summarizes all the nodes participating in a function call.
The entry points to this system are `ApplicationNode` (for in-language function
calls) and `InteropLibrary<Function>` (for polyglot function calls).
![Function Call Flow](https://user-images.githubusercontent.com/5780639/84035237-5c2d9800-a993-11ea-826d-72f3ddffcb54.png)

5
docs/runtime/instruments.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Instruments
Instruments are used to track runtime events to allow for profiling, debugging
and other kinds of behavior analysis at runtime.
@ -18,11 +19,13 @@ and other kinds of behavior analysis at runtime.
<!-- /MarkdownTOC -->
## Naming Conventions
Every Instrument must be implemented in Java and have name that ends with
`Instrument`. This requirement is to ensure that the [fix](#fixing-compilation)
described below works.
## Fixing Compilation
Annotations are used to register the implemented instruments with Graal. The
annotation processor is triggered when recompiling the Java files.
Unfortunately, when doing an incremental compilation, only the changed files are
@ -32,4 +35,4 @@ haven't been recompiled, leading to runtime errors about missing instruments.
To fix that, we add the
[`FixInstrumentsGeneration.scala`](../../project/FixInstrumentsGeneration.scala)
task which detects changes to any of the instruments and forces recompilation of
all instruments in the project by removing their classfiles.
all instruments in the project by removing their classfiles.

48
docs/runtime/runtime-features.md
View File

@ -7,8 +7,9 @@ order: 4
---
# Runtime Features
This document contains a detailed specification of Enso's runtime. It includes
a description of the technologies on which it is built, as well as the features
This document contains a detailed specification of Enso's runtime. It includes a
description of the technologies on which it is built, as well as the features
and functionality that it is required to support. In addition, the document aims
to explain why _this_ design, rather than one of the many alternatives available
to the team.
@ -53,6 +54,7 @@ then describe the design of each component in detail.
<!-- /MarkdownTOC -->
# Architectural Overview
The Enso runtime is just one of the many components of the Enso ecosystem. This
section provides an overview of how it fits into the broader ecosystem, with a
particular focus on how it enables workflows for Enso Studio, the Enso CLI, and
@ -61,6 +63,7 @@ architecture of the runtime itself, breaking down the opaque 'runtime' label
into the
## The Broader Enso Ecosystem
While the runtime is arguably the core part of Enso, for the language would not
be able to exist without it, the language's success is just as dependent on the
surrounding ecosystem.
@ -80,11 +83,11 @@ understanding how the runtime fits into the ecosystem.
- **GUI Backend:** The GUI backend is instantiated for each project, and is
responsible for all of the user-facing logic that goes into interaction with
the Enso runtime.
+ **Graph State Manager:** This component handles management of the state
- **Graph State Manager:** This component handles management of the state
required to draw the graph in the GUI.
+ **Double Representation Manager:** This component handles the encoding and
- **Double Representation Manager:** This component handles the encoding and
decoding of the Enso program to and from the intermediate representation.
+ **Undo/Redo Manager:** This component handles undo and redo for the graph, a
- **Undo/Redo Manager:** This component handles undo and redo for the graph, a
somewhat novel operation as it does not not always have a 1:1 correspondence
with textual editing.
- **CLI:** This provides a command-line (specifically a terminal) interface to
@ -97,6 +100,7 @@ understanding how the runtime fits into the ecosystem.
interfaces to foreign languages.
## The Runtime's Architecture
In order to better appreciate how the components specified below interact, it is
important to have an understanding of the high-level architecture of the runtime
itself. The design in this document pertains _only_ to the 'Enso Runtime'
@ -108,6 +112,7 @@ information between the various components.
TBC...
# Choosing GraalVM
Building the runtime on top of GraalVM was of course not the only choice that
could've been made, but it was overwhelmingly the most sensible option out of
those considered.
@ -118,13 +123,13 @@ being considered.
- **LLVM:** A battle-tested and comprehensive toolchain for the creation of
language compilers, [LLVM](https://llvm.org/) includes facilities for
compilation, optimisation, JIT, and linking.
- **GHC:** The [Glasgow Haskell Compiler](https://gitlab.haskell.org/ghc/) is
a sophisticated compiler and runtime for Haskell that provides a
- **GHC:** The [Glasgow Haskell Compiler](https://gitlab.haskell.org/ghc/) is a
sophisticated compiler and runtime for Haskell that provides a
language-agnostic set of internal representations that could be leveraged to
compile and/or interpret other functional languages.
- **JVM:** The [JVM](https://openjdk.java.net/) is a high-performance virtual
machine that includes sophisticated garbage collection, profiling tools, and
a JIT compiler.
machine that includes sophisticated garbage collection, profiling tools, and a
JIT compiler.
- **GraalVM:** A universal virtual machine and language development toolkit,
[GraalVM](https://www.graalvm.org/) provides a framework for building language
interpreters, as well as a JIT compiler. Most importantly, it provides tools
@ -142,6 +147,7 @@ upsides (e.g. the JVM's sophisticated garbage collection machinery), they all
had at least one 'fatal flaw' for Enso's use case.
### Speed of Development
A language runtime is a complex beast, so any solution that could remove some of
the implementation burden would be beneficial to Enso as a product.
@ -167,6 +173,7 @@ interface, thereby enabling rapid development of a performant runtime without
the need to implement complex components such as a GC and concurrency.
### Language Interoperability Support
With Enso aiming to be the be-all and end-all for the data-science world, the
ability to seamlessly interoperate with other programming languages is key. This
means that a user should be able to paste in some Python or R code and have it
@ -186,6 +193,7 @@ dream for ensuring that Enso can seamlessly communicate with a whole host of
other programming languages.
### Implementation Performance
Data science often involves the manipulation of very large amounts of data, and
ensuring that an interactive environment like Enso doesn't slow down as it does
so requires a high level of performance.
@ -202,6 +210,7 @@ of effort, while still providing comprehensive facilities to improve performance
further in the future.
### Maintenance Burden
Just as important as getting a working runtime is the ability for the developers
to improve and evolve it. This encompasses many factors, but Enso is primarily
concerned with being able to evolve without having to account for undue changes
@ -229,10 +238,12 @@ and as a result provides many of the facilities required by Enso for free or at
least for little effort.
# The Runtime Components
Like any sensible large software project, Enso's runtime is modular and broken
down into components. These are described in detail below.
## Language Server
The language server component is responsible for controlling the runtime itself.
It communicates with other portions of the ecosystem (such as the REPL and the
Enso Studio backend) via a protocol. While this protocol is based on the
@ -253,6 +264,7 @@ it has been extended significantly to better support Enso's use-cases.
-->
## Filesystem Driver
This component of the runtime deals with access from the runtime to external
devices. This includes the Enso code files on disk, but is also responsible for
watching filesystem resources (such as databases, files, and sockets) that are
@ -265,6 +277,7 @@ used by Enso programs.
-->
## Typechecker
The typechecker is the portion of the runtime that handles the type-inference
and type-checking of Enso code. This is a sophisticated piece of machinery, with
the primary theory under which it operates being described in the specification
@ -280,6 +293,7 @@ of [the type system](../types/README.md).
-->
## Optimiser
With much of Enso's performance relying on the JIT optimiser built into Graal,
the native language optimiser instead relies on handling more front-end specific
optimisations.
@ -296,6 +310,7 @@ optimisations.
-->
## Interpreter and JIT
The interpreter component is responsible for the actual execution of Enso code.
It is built on top of the Truffle framework provided by GraalVM, and is JIT
compiled by GraalVM.
@ -312,10 +327,12 @@ compiled by GraalVM.
-->
# Cross-Cutting Concerns
The runtime also has to deal with a number of concerns that don't fit directly
into the above components, but are nevertheless important parts of the design.
## Caching
The runtime cache for Enso is a key part of how it delivers exceptional
performance when working on big data sets. The key recognition, as seen in many
data processing tools, is that changing code or data often doesn't require the
@ -335,10 +352,11 @@ portions that are required of it, while using cached results for the rest.
-->
## Profiling and Debugging
Similarly important to the Enso user experience is the ability to visually
debug and profile programs. This component deals with the retrieval, storage,
and manipulation of profiling data, as well as the ability to debug programs in
Enso using standard and non-standard debugging paradigms.
Similarly important to the Enso user experience is the ability to visually debug
and profile programs. This component deals with the retrieval, storage, and
manipulation of profiling data, as well as the ability to debug programs in Enso
using standard and non-standard debugging paradigms.
<!-- TODO
- An analysis of how breakpoints can be set in the Truffle interpreter.
@ -350,6 +368,7 @@ Enso using standard and non-standard debugging paradigms.
-->
## Foreign Language Interoperability
This component deals with using the GraalVM language interoperability features
to provide a seamless interface to foreign code from inside Enso.
@ -362,6 +381,7 @@ to provide a seamless interface to foreign code from inside Enso.
-->
## Lightweight Concurrency
Though not strictly a component, this section deals with how Enso can provide
its users with lightweight concurrency primitives in the form of green threads.
@ -374,6 +394,7 @@ its users with lightweight concurrency primitives in the form of green threads.
-->
# The Initial Version of the Runtime
In order to have a working version of the new runtime as quickly as possible, it
was decided to design and build an initial, stripped-down version of the final
design. This design focused on development of a minimal working subset of the
@ -385,6 +406,7 @@ runtime that would allow Enso to run.
-->
# Development Considerations
As part of developing the new Enso runtime, the following things need to be
accounted for. This is to ensure that the eventual quality of the software is
high, and that we also provide a product that is actually useful to our users.

80
docs/runtime/searcher.md
View File

@ -7,6 +7,7 @@ order: 8
---
# Searcher
The language auto-completion feature requires an ability to search the codebase
using different search criteria. This document describes the Searcher module,
which consists of the suggestions database and the ranking algorithm for
@ -23,69 +24,76 @@ ordering the results.
<!-- /MarkdownTOC -->
## Suggestions Database
Suggestions database holds entries for fulfilling the auto-completion requests
with additional indexes used to answer the different search criteria. The
database is populated partially by analyzing the sources and enriched with the
extra information from the runtime.
API data types for the suggestions database are specified in the [Language
Server Message Specification](../language-server/protocol-language-server.md)
API data types for the suggestions database are specified in the
[Language Server Message Specification](../language-server/protocol-language-server.md)
document.
### Database Structure
Implementation utilizes the SQLite database.
Database is created per project and the database file stored in the project
directory. That way the index can be preserved between the IDE restarts.
#### Suggestions Table
Suggestions table stores suggestion entries.
| Column | Type | Description |
| --- | --- | --- |
| `id` | `INTEGER` | the unique identifier |
| `externalId` | `UUID` | the external id from the IR |
| `kind` | `INTEGER` | the type of suggestion entry, i.e. Atom, Method, Function, or Local |
| `module` | `TEXT` | the module name |
| `name` | `TEXT` | the suggestion name |
| `self_type` | `TEXT` | the self type of the Method |
| `return_type` | `TEXT` | the return type of the entry |
| `scope_start` | `INTEGER` | the start position of the definition scope |
| `scope_end` | `INTEGER` | the end position of the definition scope |
| `documentation` | `TEXT` | the documentation string |
| Column | Type | Description |
| --------------- | --------- | ------------------------------------------------------------------- |
| `id` | `INTEGER` | the unique identifier |
| `externalId` | `UUID` | the external id from the IR |
| `kind` | `INTEGER` | the type of suggestion entry, i.e. Atom, Method, Function, or Local |
| `module` | `TEXT` | the module name |
| `name` | `TEXT` | the suggestion name |
| `self_type` | `TEXT` | the self type of the Method |
| `return_type` | `TEXT` | the return type of the entry |
| `scope_start` | `INTEGER` | the start position of the definition scope |
| `scope_end` | `INTEGER` | the end position of the definition scope |
| `documentation` | `TEXT` | the documentation string |
#### Arguments Table
Arguments table stores all suggestion arguments with `suggestion_id` foreign
key.
| Column | Type | Description |
| --- | --- | --- |
| `id` | `INTEGER` | the unique identifier |
| `suggestion_id` | `INTEGER` | the suggestion key this argument relates to |
| `index` | `INTEGER` | the argument position in the arguments list |
| `name` | `TEXT` | the argument name |
| `type` | `TEXT` | the argument type; const 'Any' is used to specify generic types |
| `is_suspended` | `INTEGER` | indicates whether the argument is lazy |
| `has_defult` | `INTEGER` | indicates whether the argument has default value |
| `default_value` | `TEXT` | the optional default value |
| Column | Type | Description |
| --------------- | --------- | --------------------------------------------------------------- |
| `id` | `INTEGER` | the unique identifier |
| `suggestion_id` | `INTEGER` | the suggestion key this argument relates to |
| `index` | `INTEGER` | the argument position in the arguments list |
| `name` | `TEXT` | the argument name |
| `type` | `TEXT` | the argument type; const 'Any' is used to specify generic types |
| `is_suspended` | `INTEGER` | indicates whether the argument is lazy |
| `has_defult` | `INTEGER` | indicates whether the argument has default value |
| `default_value` | `TEXT` | the optional default value |
#### Suggestions Version Table
Versions table has a single row with the current database version. The version
is updated on every change in the suggestions table.
| Column | Type | Description |
| --- | --- | --- |
| `id` | `INTEGER` | the unique identifier representing the currend database version |
| Column | Type | Description |
| ------ | --------- | --------------------------------------------------------------- |
| `id` | `INTEGER` | the unique identifier representing the currend database version |
#### File Versions Table
Keeps track of SHA versions of the opened files.
| Column | Type | Description |
| --- | --- | --- |
| `path` | `TEXT` | the unique identifier of the file |
| Column | Type | Description |
| -------- | ------ | --------------------------------- |
| `path` | `TEXT` | the unique identifier of the file |
| `digest` | `BLOB` | the SHA hash of the file contents |
### Static Analysis
The database is filled by analyzing the Intermediate Representation (`IR`)
parsed from the source.
@ -102,6 +110,7 @@ information specifies the scope where the node was defined, which is used in the
results [ranking](#results-ranking) algorithm.
### Runtime Inspection
The `IR` is missing information about the types. Runtime instrumentation is used
to capture the types and refine the database entries.
@ -110,6 +119,7 @@ Information for suggestions database gathered in runtime:
- Types
### Search Indexes
The database allows searching the entries by the following criteria, applying
the [ranking](#results-ranking) algorithm:
@ -119,13 +129,15 @@ the [ranking](#results-ranking) algorithm:
- documentation tags
## Results Ranking
The search result has an intrinsic ranking based on the scope and the type.
### Scope
The results from the local scope have the highest rank in the search result. As
the scope becomes wider, the rank decreases.
``` ruby
```ruby
const_x = 42
main =
@ -139,10 +151,12 @@ For example, when completing the argument of `calculate: Number -> Number`
function, the search results will have the order of: `x2` > `x1` > `const_x`.
### Type
Suggestions based on the type are selected by matching with the string runtime
type representation.
## Implementation
The searcher primarily consists of:
- Suggestions database that stores suggestion entries and SHA hashes of the
@ -185,6 +199,7 @@ The searcher primarily consists of:
```
### Indexing
Indexing is a process of extracting suggestions information from the `IR`. To
keep the index in the consistent state, the language server tracks SHA versions
of all opened files in the suggestions database.
@ -207,12 +222,14 @@ of all opened files in the suggestions database.
module from the database and update the users.
### Suggestion Builder
The suggestion builder takes part in the indexing process. It extracts
suggestions from the compiled `IR`. It also converts `IR` locations represented
as an absolute char indexes (from the beginning of the file) to a position
relative to a line used in the rest of the project.
### Runtime Instrumentation
Apart from the type ascriptions, we can only get the return types of expressions
in runtime. On the module execution, the language server listens to the
`ExpressionValuesComputed` notifications sent from the runtime, which contain
@ -220,6 +237,7 @@ the external id and the actual return type. Then the language server uses
external id as a key to update return types in the database.
### Search Requests
The search request handler has direct access to the database. The completion
request is more complicated then others. To query the database, it needs to
convert the requested file path to the module name. To do this, on startup, the

18
docs/runtime/unbounded-recursion.md
View File

@ -7,6 +7,7 @@ order: 5
---
# Unbounded Recursion
The JVM (and hence, GraalVM) do not have support for segmented stacks, and hence
do not allow for computation of unbounded recursion - if you make too many
recursive function calls you can cause your stack to overflow. Quite obviously,
@ -45,6 +46,7 @@ use of unbounded recursion in Enso on GraalVM.
<!-- /MarkdownTOC -->
## A Baseline
In an ideal world, we'd like the performance of Enso's recursive calls to
approximate that of Haskell, which can be made to have fairly optimal
performance for a functional language. Basic measurements for a Haskell program
@ -60,11 +62,13 @@ perform the same summation of integers. They have a variable threshold, listed
in the results as `inputSize`.
## Emulating Stack Segmentation with Threads
As each new thread has its own stack, we can exploit this to emulate the notion
of split stacks as used in many functional programming languages. The basic idea
is to work out when you're about to run out of stack space,
### When to Spawn a Thread
One of the main problems with this approach is that you want to make as much use
out of the stack for a given thread as possible. However, it is very difficult
to get an accurate idea of when a stack may be _about_ to overflow. There are
@ -82,6 +86,7 @@ permits as a valid stack implementation. This means that from a specification
perspective there is very little that could actually be relied upon.
### Conservative Counting
A naive and obvious solution is to maintain a counter that tracks the depth of
your call stack. This would allow you to make a conservative estimate of the
amount of stack you have remaining, and spawn a new thread at some threshold.
@ -112,6 +117,7 @@ thread are approximately equal to Haskell. It is hence possible that a method
that reduces the cost of context switching could make this approach feasible.
### Catching the Overflow
Though it is heavily recommended against by the Java documentation, it is indeed
possible to catch the `StackOverflowError`. While this provides accurate info
about when you run out of stack space, it has one major problem: you may not
@ -153,11 +159,13 @@ threads and hence reduce the context switching overhead. Nevertheless, this is
still very slow compared to Haskell baseline.
### Thread Pools
While a thread pool is conventionally seen as a way to amortise the cost of
spawning threads, this approach to recursion requires far more threads than is
really feasible to keep around in a pool, so we've not explored that approach.
### Project Loom
If project loom's coroutines and / or fibres were stable, these would likely
help somewhat by reducing the thread creation overhead that is primarily down to
OS-level context switches.
@ -167,6 +175,7 @@ it is not only far from stable, but also has no guarantee that it will actually
make it into the JVM.
## Avoiding Stack Usage via a CPS Transform
Transforming recursive calls into CPS allows us to avoid the _need_ for using
the stack instead of trying to augment it. This could be implemented as a global
transformation, or as a local one only for recursive calls.
@ -188,6 +197,7 @@ size grows to the point that additional stack segments are needed, the execution
performance is within spitting distance of the Haskell code.
### The CPS Transform
While it is tempting to perform the CPS transform globally for the whole
program, this has some major drawbacks:
@ -204,6 +214,7 @@ program stack in a thread-safe manner and perform the transformation at runtime
(e.g. `private static ThreadLocal<Boolean> isExecuting;`).
### A Hybrid Approach
As we clearly don't want to CPS transform the program globally, we need some
mechanism by which we can rewrite only when necessary. As discussed above, we
could do this via a dynamic runtime analysis, but we could also potentially make
@ -239,6 +250,7 @@ is trivial, but it may require some more sophisticated tracing in the case of
mutually-recursive functions.
## Linearised Representations
While not something that we could feasibly do at the moment, one of the
potential solutions for this is to statically compile the language to a
linearised representation. Rather than trying to implement the CPS transform in
@ -246,8 +258,8 @@ a Truffle interpreter not designed for it, we could instead compile Enso to a
low-level IR format which has no stack frames, and instead just uses jumps.
Whether we write this IR ourselves or use an existing one implemented as a
Truffle language, such as WASM bytecode (currently very experimental) or LLVM
IR (much more tried and tested), this would provide a number of benefits:
Truffle language, such as WASM bytecode (currently very experimental) or LLVM IR
(much more tried and tested), this would provide a number of benefits:
- The IR output by the compiler phase need not be fed into the truffle
interpreter for said IR.
@ -264,12 +276,14 @@ However, such an approach also has some major downsides:
which, is currently tied quite strongly into the Truffle language life-cycle.
## Alternatives
At the current time there are no apparent alternatives to the three approaches
discussed above. While it would be ideal for the JVM to have native support for
stack segmentation on the heap, this would likely be an in-depth and significant
amount of work to add, with no guarantee that it would be accepted into main.
## Open Questions
The following are questions for which we don't yet have answers:
- Are there any ways to instrument a JVM thread to detect when it's about to

7
docs/semantics/README.md
View File

@ -7,10 +7,11 @@ order: 0
---
# Enso's Semantics
Much like we have specifications for the [syntax](../syntax/README.md) and the
[type system](../types/README.md) in Enso, we also need a specification of the
language semantics. These documents specify the executable semantics of the
Enso language.
language semantics. These documents specify the executable semantics of the Enso
language.
> The actionables for this section are:
>
@ -24,7 +25,7 @@ This specification is broken down into the following sections:
compiler diagnostics.
- [**Dispatch:**](./dispatch.md) The semantics of Enso's dispatch system.
- [**Errors:**](./errors.md) The semantics of Enso's various error systems.
- [**Evaluation:**](./evaluation.md) Enso's evaluation semantics, including
- [**Evaluation:**](./evaluation.md) Enso's evaluation semantics, including
those of suspended computations.
- [**Modules:**](./modules.md) The semantics of Enso's modules.
- [**Scoping:**](./scoping.md) Enso's scoping and identifier resolution rules.

6
docs/semantics/bindings.md
View File

@ -7,8 +7,9 @@ order: 1
---
# Bindings
A "binding" is a portion of and Enso program that creates a new name and binds
a value to that name.
A "binding" is a portion of and Enso program that creates a new name and binds a
value to that name.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -17,6 +18,7 @@ a value to that name.
<!-- /MarkdownTOC -->
## Binding Return Value
While some expression-based languages with bindings have the binding return the
value assigned to the binding, we feel that this is far too error prone.
Consider the following code as a demonstration:

7
docs/semantics/diagnostics.md
View File

@ -7,9 +7,10 @@ order: 2
---
# Diagnostics
Due to the highly interactive and always-online nature of a running Enso
program, it is very important that compile diagnostics (such as lints, errors,
and warnings) are surfaced in the _running_ language. This is particularly
and warnings) are surfaced in the _running_ language. This is particularly
important for visual mode.
To this end, Enso provides mechanisms by which diagnostics are scoped to the
@ -26,13 +27,13 @@ nodes in the interpreter so that they can be interacted with at runtime.
## Warnings at Runtime
> The actionables for this section are:
>
>
> - Specify how warnings behave at runtime.
## Errors at Runtime
> The actionables for this section are:
>
>
> - Specify how errors behave at runtime.
> - Talk about their limited scope and how they can be interacted with at
> runtime.

5
docs/semantics/dispatch.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Dispatch
Enso has support for functions that are both statically and dynamically
dispatched. We call the first ones _functions_, while the latter are referred to
as _methods_.
@ -21,11 +22,11 @@ as _methods_.
## Static Dispatch
> The actionables for this section are:
>
>
> - Specify the semantics of static dispatch in Enso.
## Dynamic Dispatch
> The actionables for this section are:
>
>
> - Specify the semantics of dynamic dispatch in Enso.

5
docs/semantics/errors.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Errors
Due to its hybrid nature, Enso supports two main exception systems that help
users to deal with errors at runtime in the language.
@ -20,11 +21,11 @@ users to deal with errors at runtime in the language.
## Asynchronous Exceptions
> The actionables for this section are:
>
>
> - Specify the semantics of Enso's async exceptions.
## Broken Values
> The actionables for this section are:
>
>
> - Specify the semantics of Enso's broken values.

3
docs/semantics/evaluation.md
View File

@ -7,6 +7,7 @@ order: 5
---
# Evaluation Semantics in Enso
Enso's evaluation semantics can be succinctly described as 'strict, but with
optional laziness'. By default, expressions in Enso are evaluated strictly, but
the programmer may choose to 'suspend' computations, and instead evaluate them
@ -20,6 +21,7 @@ at the point they are needed.
<!-- /MarkdownTOC -->
## Strict Evaluation
Though Enso shares many syntactic similarities with Haskell, the most famous
example of a lazily evaluated language, Enso is not lazy. Instead, Enso is a
language that is strict.
@ -35,6 +37,7 @@ language that is strict.
> - Make this far better specified.
## Optional Suspension
Laziness, however, can often be quite useful for defining certain kinds of API.
To that end, Enso provides support for optional laziness, more specifically
optional _suspension_, through the built-in `Suspended` type.

5
docs/semantics/modules.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Modules
Enso has a rudimentary module system in order to aid code modularity.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
@ -19,11 +20,11 @@ Enso has a rudimentary module system in order to aid code modularity.
## Enso Modules
> The actionables for this section are:
>
>
> - Specify the current state of the Enso module system.
## Future Directions
> The actionables for this section are:
>
>
> - Specify what we'd like to do with modules in the future.

14
docs/semantics/scoping.md
View File

@ -7,6 +7,7 @@ order: 7
---
# Scoping Rules
Enso's scoping rules should be fairly familiar to those coming from other
languages that are immutable (or make heavy use of immutability). In essence,
Enso is a lexically-scoped language where bindings may be shadowed in child
@ -24,6 +25,7 @@ scopes.
<!-- /MarkdownTOC -->
## Scopes
A scope is the span in the code within which a set of accessible identifiers
occurs. A nested scope may:
@ -54,6 +56,7 @@ valid entity," and hence implies "can have its value used."
> contexts, we need to revisit the above rules.
## Introducing New Scopes
The following constructs introduce new scopes in Enso:
- **Modules:** Each module (file) introduces a new scope.
@ -88,6 +91,7 @@ the above constructs:
> implementation of `case` evolves.
### Scoping of Type Signatures
Currently, type signatures in Enso obey a simple set of typing rules:
- The RHS of the type ascription introduces a new scope that is a child of the
@ -97,20 +101,23 @@ Currently, type signatures in Enso obey a simple set of typing rules:
>
> In order to enable much of the flexible metaprogramming ability that Enso aims
> for, we have an additional set of scoping rules for type signatures:
>
> - Both operands of the type ascription operator share a scope.
> - If two names are used on the type and term levels to refer to the same entity,
> both are valid but this issues a warning. Referring to the same entity means
> that they are two names for the same underlying object.
> - If two names are used on the type and term levels to refer to the same
> entity, both are valid but this issues a warning. Referring to the same
> entity means that they are two names for the same underlying object.
> - Name clashes are disallowed unless the clashing names refer to the same
> entity.
> - Do we actually want to support this?
> - What complexities does this introduce wrt typechecking?
## Implementation Notes
This section contains notes on the implementation of the Enso scoping rules in
the interpreter.
### Function Call Arguments
In order to support suspended function arguments in the interpreter in a
performant way, we implicitly wrap _all_ function arguments in a suspension. In
conjunction with making the function itself responsible for when its arguments
@ -127,6 +134,7 @@ However, it _does_ require creating a small hack in the Alias Analysis process:
To this end, we account for this implementation detail in alias analysis.
### Collapsing Scopes
Another quirk of the internal alias analysis process is down to the fact that
the Enso IR represents Methods, functions, and blocks as separate constructs.
This means that if you had a method containing a function containing a block, a

6
docs/style-guide/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso Developer Style Guides
This folder contains the style guides for developers working on the various
languages that make up the Enso project.
@ -14,3 +15,8 @@ languages that make up the Enso project.
- [Java](./java.md)
- [Rust](./rust.md)
- [Haskell](./haskell.md)
Our configuration and documentation also has style guides:
- [Markdown](./markdown.md)
- [YAML](./yaml.md)

222
docs/style-guide/haskell.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Haskell Style Guide
Like many style guides, this Haskell style guide exists for two primary reasons.
The first is to provide guidelines that result in a consistent code style across
all of the Enso codebases, while the second is to guide people towards a style
@ -46,10 +47,12 @@ programmer burden; there is usually only _one way_ to lay out code correctly.
<!-- /MarkdownTOC -->
## Code Formatting
This section explains the rules for visually laying out your code. They provide
a robust set of guidelines for creating a consistent visual to the code.
### Whitespace
The rules for whitespace in the Enso codebases are relatively simple:
- 4 spaces are used for indentation, with no tabs.
@ -58,6 +61,7 @@ The rules for whitespace in the Enso codebases are relatively simple:
[alignment](#alignment) as discussed below.
### Line Wrapping
In order to provide visual consistency across our codebases, and also to
contribute to making our code easier to scan, we enforce that all code should be
wrapped to 80 characters width at a maximum.
@ -106,8 +110,8 @@ break lines. We use the following guidelines:
veryLongFunction2 veryLongArgument2 veryLongArgument3
```
- Function signatures should wrap on the `=>` and `->`, and in the context of
a doc comment should have each argument on a separate line.
- Function signatures should wrap on the `=>` and `->`, and in the context of a
doc comment should have each argument on a separate line.
- Lists (and all list-like constructs e.g. constraint tuples, import lists)
should be wrapped with a _leading_ comma, aligned with the opening bracket,
and a space between the opening bracket and the first item. This is also used
@ -135,6 +139,7 @@ line. We would much prefer that the code wraps to two lines and that naming
remains intelligible than names become so shortened as to be useless.
### Alignment
When there are multiple lines that are visually similar, we try to align the
similar portions of the lines vertically.
@ -153,6 +158,7 @@ multiple lines, it _no longer counts_ as visually similar, and hence subsequent
lines should not be aligned with it.
### Naming
Enso has some fairly simple general naming conventions, though the sections
below may provide more rules for use in specific cases.
@ -172,6 +178,7 @@ below may provide more rules for use in specific cases.
- Naming should use American English spelling.
### Imports
Organising imports properly means that it's easy to find the provenance of a
given function even in the absence of IDE-style tooling. We organise our imports
in four sections, each of which may be omitted if empty.
@ -229,6 +236,7 @@ import Vector (Vector (Vector), test)
```
### Exports
There is nothing more frustrating than having a need to use a function in a
module that hasn't been exported. To that end, we do not allow for restricted
export lists in our modules.
@ -240,6 +248,7 @@ that these functions can be imported by clients of the API if they need to, but
that we provide no guarantees about API stability when using those functions.
### Section Headers
In order to visually break up the code for easier 'visual grepping', we organise
it using section headers. These allow us to easily find the section that we are
looking for, even in a large file.
@ -270,6 +279,7 @@ subsections may be omitted if they don't exist, and a file may contain multiple
of these sections as relevant.
### Auto-Formatting
While we have attempted to use haskell auto-formatters to enforce many of the
above stylistic choices in this document, none have been found to be flexible
enough for our needs. However, as tools evolve or new ones emerge, we are open
@ -277,6 +287,7 @@ to revisiting this decision; if you know of a tool that would let us automate
the above stylistic rules, then please speak up.
## Commenting
Comments in code are a tricky area to get right as we have found that comments
often expire quickly, and in absence of a way to validate them, remain incorrect
for long periods of time. In order to best deal with this problem, we make the
@ -300,6 +311,7 @@ Code should be written in such a way that it guides you over what it does, and
comments should not be used as a crutch for badly-designed code.
### Documentation Comments
One of the primary forms of comment that we allow across the Enso codebases is
the doc comment. Every language construct that can have an associated doc
comment should do so. These are intended to be consumed by users of the API, and
@ -314,6 +326,7 @@ choices made in the function's implementation. See [Source Notes](#source-notes)
below for how to indicate that kind of information.
### Source Notes
Source Notes is a mechanism for moving detailed design information about a piece
of code out of the code itself. In doing so, it retains the key information
about the design while not impeding the flow of the code.
@ -387,6 +400,7 @@ but can also be used for:
its usage is safe in this context.
### TODO Comments
We follow a simple convention for `TODO` comments in our codebases:
- The line starts with `TODO` or `FIXME`.
@ -402,6 +416,7 @@ For example:
```
### Other Comment Usage
There are, of course, a few other situations where commenting is very useful:
- **Commenting Out:** You may comment out code while developing it, but if you
@ -412,10 +427,12 @@ There are, of course, a few other situations where commenting is very useful:
where the bug has been reported.
## Program Design
Any good style guide goes beyond purely stylistic rules, and also talks about
design styles to use in code.
### Libraries
The Enso project has many internal libraries that are useful, but we have found
that maintaining these on Hackage while they are under such active development
is counterproductive.
@ -426,6 +443,7 @@ freely used by others of our projects by depending on a git commit of the
project that they live in. All of these are safe to use.
#### Prologue
`Prologue` is our replacement for Haskell's `Prelude`. For the most part it is
compatible with the prelude, though it is designed with a safe API as the first
port of call.
@ -443,6 +461,7 @@ computation.
It is highly recommended that you scan the code of Prologue.
#### Safety
It is incredibly important that we can trust the code that we use, and hence we
tend to disallow the definition of unsafe functions in our public API. When
defining an unsafe function, you must account for the following:
@ -458,11 +477,13 @@ defining an unsafe function, you must account for the following:
Furthermore, we do not allow for code containing pattern matches that can fail.
#### Control.Monad.Exception
We have our own exception framework based on `ExceptT` that encodes exception
usage at the type level. This ensures that all synchronous exceptions must be
dealt with.
It is defined in [`lib/exception/`](https://github.com/enso-org/luna/tree/master/lib/exception)
It is defined in
[`lib/exception/`](https://github.com/enso-org/luna/tree/master/lib/exception)
and contains utilities for declaring that a function throws an exception, as
well as throwing and catching exceptions.
@ -485,6 +506,7 @@ so they should use the following guidelines:
re-wrapping an error thrown inside our function.
### Modules
Unlike much of the Haskell ecosystem, we tend to design modules to be imported
_qualified_ rather than unqualified. This means that we have a few rules to keep
in mind:
@ -499,13 +521,15 @@ in mind:
`State.run` rather than `State.runState`).
### Data Declarations
When declaring data types in the Enso codebases, please make sure to keep the
following rules of thumb in mind:
- For single-constructor types:
+ Write the definition across multiple lines.
+ Always name your fields.
+ Always generate lenses.
- Write the definition across multiple lines.
- Always name your fields.
- Always generate lenses.
```hs
data Rectangle = MkRectangle
@ -514,10 +538,12 @@ following rules of thumb in mind:
} deriving (Eq, Ord, Show)
makeLenses ''Rectangle
```
- For multiple-constructor data-types:
+ Write the definition across multiple lines.
+ Never name your fields.
+ Generate prisms only when necessary.
- Write the definition across multiple lines.
- Never name your fields.
- Generate prisms only when necessary.
```hs
data Shape
@ -528,17 +554,18 @@ following rules of thumb in mind:
- Always prefer named fields over unnamed ones. You should only use unnamed
fields if one or more of the following hold:
+ Your data type is one where you are are _sure_ that separate field access
- Your data type is one where you are are _sure_ that separate field access
will never be needed.
+ You are defining a multiple-constructor data type.
- You are defining a multiple-constructor data type.
- Sort deriving clauses in alphabetical order, and derive the following for your
type if logically correct:
+ General Types: `Eq`, `Generic`, `NFData`, `Ord`, `Show`.
+ Parametric 1-Types: `Applicative`, `Alternative`, `Functor`.
+ Monads: `Monad`, `MonadFix`.
+ Monad Transformers: `MonadTrans`.
- General Types: `Eq`, `Generic`, `NFData`, `Ord`, `Show`.
- Parametric 1-Types: `Applicative`, `Alternative`, `Functor`.
- Monads: `Monad`, `MonadFix`.
- Monad Transformers: `MonadTrans`.
#### Lenses
The Enso codebases make significant use of Lenses, and so we have some rules for
their use:
@ -570,6 +597,7 @@ This will generate lenses with names like `vector_x`, `vector_y`, and `point_x`,
`point_y`.
### Testing and Benchmarking
New code should always be accompanied by tests. These can be unit, integration,
or some combination of the two, and they should always aim to test the new code
in a rigorous fashion.
@ -597,14 +625,16 @@ _Do not benchmark a development build_ as the data you get will often be
entirely useless.
### Warnings, and Lints
In general, we aim for a codebase that is free of warnings and lints, and we do
this using the following ideas:
#### Warnings
New code should introduce no new warnings onto main. You may build with
warnings on your own branch, but the code that is submitted as part of a PR
should not introduce new warnings. You should also endeavour to fix any warnings
that you come across during development.
New code should introduce no new warnings onto main. You may build with warnings
on your own branch, but the code that is submitted as part of a PR should not
introduce new warnings. You should also endeavour to fix any warnings that you
come across during development.
Sometimes it is impossible to fix a warning (e.g. TemplateHaskell generated code
often warns about unused pattern matches). In such cases, you are allowed to
@ -613,15 +643,17 @@ must be accompanied by a source note explaining _why_ the warning cannot be
fixed otherwise.
#### Lints
We also recommend using HLint on your code as a stylistic guide, as we find that
its suggestions in general lead to more readable code. If you don't know how to
set up automatic linting for your editor, somebody will be able to help.
An example of an anti-pattern that HLint will catch is the repeated-`$`. Instead
of `foo $ bar $ baz $ bam quux`, you should write `foo . bar. baz $ bam quux`
to use function composition.
of `foo $ bar $ baz $ bam quux`, you should write `foo . bar. baz $ bam quux` to
use function composition.
## Language Extensions
Much like any sophisticated Haskell codebase, Enso makes heavy use of the GHC
language extensions. We have a broad swath of extensions that are enabled by
default across our projects, and a further set which are allowed whenever
@ -639,6 +671,7 @@ available to you, we recommend checking the GHC Users Guide entry for that
extension (linked from the extension's table below).
### Default Extensions
The following language extensions are considered to be so safe, or to have such
high utility, that they are considered to be Enso's set of default extensions.
You can find said set of extensions for Enso itself defined in a
@ -647,7 +680,7 @@ You can find said set of extensions for Enso itself defined in a
#### AllowAmbiguousTypes
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`AllowAmbiguousTypes`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-AllowAmbiguousTypes) |
| **Flag** | `-XAllowAmbiguousTypes` |
@ -662,7 +695,7 @@ would not be possible without `-XAllowAmbiguousTypes`.
#### ApplicativeDo
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`ApplicativeDo`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-ApplicativeDo) |
| **Flag** | `-XApplicativeDo` |
@ -670,14 +703,14 @@ This extension allows desugaring of do-notation based on applicative operations
(`<$>`, `<*>`, and `join`) as far as is possible. This will preserve the
original semantics as long as the type has an appropriate applicative instance.
Applicative operations are often easier to optimise than monadic ones, so if
you can write a computation using applicatives please do. This is the same
reason that we prefer `pure` to `return`.
Applicative operations are often easier to optimise than monadic ones, so if you
can write a computation using applicatives please do. This is the same reason
that we prefer `pure` to `return`.
#### BangPatterns
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`BangPatterns`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-BangPatterns) |
| **Flag** | `-XBangPatterns` |
@ -689,7 +722,7 @@ laziness.
#### BinaryLiterals
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`BinaryLiterals`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-BinaryLiterals) |
| **Flag** | `-XBinaryLiterals` |
@ -699,7 +732,7 @@ This can be very useful when writing bit-masks, and other low-level code.
#### ConstraintKinds
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`ConstraintKinds`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-ConstraintKinds) |
| **Flag** | `-XConstraintKinds` |
@ -713,7 +746,7 @@ All of these are very useful.
#### DataKinds
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DataKinds`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DataKinds) |
| **Flag** | `-XDataKinds` |
@ -727,7 +760,7 @@ makes the Enso codebase so expressive and yet so safe.
#### DefaultSignatures
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DefaultSignatures`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DefaultSignatures) |
| **Flag** | `-XDefaultSignatures` |
@ -739,7 +772,7 @@ implementation of a typeclass method.
#### DeriveDataTypeable
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DeriveDataTypeable`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DeriveDataTypeable) |
| **Flag** | `-XDeriveDataTypeable` |
@ -750,7 +783,7 @@ type representations with types. This is often useful for low-level programming.
#### DeriveFoldable
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DeriveFoldable`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DeriveFoldable) |
| **Flag** | `-XDeriveFoldable` |
@ -761,7 +794,7 @@ kind `Type -> Type`.
#### DeriveFunctor
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DeriveFunctor`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DeriveFunctor) |
| **Flag** | `-XDeriveFunctor` |
@ -771,7 +804,7 @@ with kind `Type -> Type`.
#### DeriveGeneric
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DeriveGeneric`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DeriveGeneric) |
| **Flag** | `-XDeriveGeneric` |
@ -782,7 +815,7 @@ generic programming.
#### DeriveTraversable
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DeriveTraversable`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DeriveTraversable) |
| **Flag** | `-XDeriveTraversable` |
@ -793,7 +826,7 @@ that can be traversed. It is a valid derivation for any data type with kind
#### DerivingStrategies
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DerivingStrategies`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DerivingStrategies) |
| **Flag** | `-XDerivingStrategies` |
@ -807,7 +840,7 @@ link.
#### DerivingVia
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DerivingVia`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DerivingVia) |
| **Flag** | `-XDerivingVia` |
@ -819,7 +852,7 @@ requires the specification of another type (the via-type) to coerce through.
#### DuplicateRecordFields
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`DuplicateRecordFields`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-DuplicateRecordFields) |
| **Flag** | `-XDuplicateRecordFields` |
@ -830,7 +863,7 @@ above in the section on [lenses](#lenses).
#### EmptyDataDecls
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`EmptyDataDecls`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-EmptyDataDecls) |
| **Flag** | `-XEmptyDataDecls` |
@ -841,7 +874,7 @@ properties in types through the use of rich kinds.
#### FlexibleContexts
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`FlexibleContexts`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-FlexibleContexts) |
| **Flag** | `-XFlexibleContexts` |
@ -851,7 +884,7 @@ that anything with kind `Constraint` is usable in a class declaration's context.
#### FlexibleInstances
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`FlexibleInstances`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-FlexibleInstances) |
| **Flag** | `-XFlexibleInstances` |
@ -862,7 +895,7 @@ support rich type-level programming.
#### Functional Dependencies
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`FunctionalDependencies`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-FunctionalDependencies) |
| **Flag** | `-XFunctionalDependencies` |
@ -877,7 +910,7 @@ Type Families.
#### GeneralizedNewtypeDeriving
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`GeneralizedNewtypeDeriving`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-GeneralizedNewtypeDeriving) |
| **Flag** | `-XGeneralizedNewtypeDeriving` |
@ -888,7 +921,7 @@ has been somewhat superseded by `-XDerivingVia`
#### InstanceSigs
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`InstanceSigs`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-InstanceSigs) |
| **Flag** | `-XInstanceSigs` |
@ -899,17 +932,17 @@ the signature provided in the class definition.
#### LambdaCase
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`LambdaCase`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-LambdaCase) |
| **Flag** | `-XLambdaCase` |
Enables `\case` as an alternative to `case <...> of`. This often results in
much cleaner code.
Enables `\case` as an alternative to `case <...> of`. This often results in much
cleaner code.
#### LiberalTypeSynonyms
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`LiberalTypeSynonyms`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-LiberalTypeSynonyms) |
| **Flag** | `-XLiberalTypeSynonyms` |
@ -920,7 +953,7 @@ context of type-level programming constructs.
#### MonadComprehensions
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`GeneralizedNewtypeDeriving`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-GeneralizedNewtypeDeriving) |
| **Flag** | `-XGeneralizedNewtypeDeriving` |
@ -930,7 +963,7 @@ any type that is an instance of `Monad`.
#### MultiParamTypeClasses
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`MultiParamTypeClasses`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-MultiParamTypeClasses) |
| **Flag** | `-XMultiParamTypeClasses` |
@ -941,7 +974,7 @@ typeclasses.
#### MultiWayIf
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`MultiWayIf`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-MultiWayIf) |
| **Flag** | `-XMultiWayIf` |
@ -949,11 +982,13 @@ This extension allows GHC to accept conditional expressions with multiple
branches, using the guard-style notation familiar from function definitions.
#### NamedWildCards
#### NegativeLiterals
#### NoImplicitPrelude
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`NoImplicitPrelude`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-NoImplicitPrelude) |
| **Flag** | `-XNoImplicitPrelude` |
@ -964,7 +999,7 @@ to use `Prologue`, our own custom prelude (discussed in the section on
#### NumDecimals
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`NumDecimals`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-NumDecimals) |
| **Flag** | `-XNumDecimals` |
@ -973,7 +1008,7 @@ Enables writing integer literals using exponential syntax.
#### OverloadedLabels
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`OverloadedLabels`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-OverloadedLabels) |
| **Flag** | `-XOverloadedLabels` |
@ -983,7 +1018,7 @@ both on its literal text and its kind. This is similar to `-XOverloadedStrings`.
#### OverloadedStrings
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`OverloadedStrings`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-OverloadedStrings) |
| **Flag** | `-XOverloadedStrings` |
@ -994,7 +1029,7 @@ can be used to represent any type that is an instance of `IsString`.
#### PatternSynonyms
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`PatternSynonyms`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-PatternSynonyms) |
| **Flag** | `-XPatternSynonyms` |
@ -1006,7 +1041,7 @@ useful for defining clean APIs to not-so-clean data.
#### QuasiQuotes
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`QuasiQuotes`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-QuasiQuotes) |
| **Flag** | `-XQuasiQuotes` |
@ -1017,7 +1052,7 @@ in Haskell source files.
#### RankNTypes
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`RankNTypes`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-RankNTypes) |
| **Flag** | `-XRankNTypes` |
@ -1028,7 +1063,7 @@ for defining clean and safe APIs.
#### RecursiveDo
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`RecursiveDo`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-RecursiveDo) |
| **Flag** | `-XRecursiveDo` |
@ -1039,7 +1074,7 @@ defined, much as for an ordinary `let`-expression.
#### ScopedTypeVariables
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`ScopedTypeVariables`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-ScopedTypeVariables) |
| **Flag** | `-XScopedTypeVariables` |
@ -1050,7 +1085,7 @@ scope of this variables is extended to the function body.
#### StandaloneDeriving
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`StandaloneDeriving`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-StandaloneDeriving) |
| **Flag** | `-XStandaloneDeriving` |
@ -1061,7 +1096,7 @@ need to create orphan instances, or to derive some non-default classes.
#### Strict
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------- |
| **Name** | [`Strict`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-Strict) |
| **Flag** | `-XStrict` |
@ -1076,7 +1111,7 @@ When disabling strict for a module using `-XNoStrict`, you also need to add
#### StrictData
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`StrictData`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-StrictData) |
| **Flag** | `-XStrictData` |
@ -1086,7 +1121,7 @@ be explicitly disabled in contexts where the strictness is undesirable.
#### TemplateHaskell
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TemplateHaskell`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TemplateHaskell) |
| **Flag** | `-XTemplateHaskell` |
@ -1097,7 +1132,7 @@ arbitrary input.
#### TupleSections
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TupleSections`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TupleSections) |
| **Flag** | `-XTupleSections` |
@ -1107,7 +1142,7 @@ extension enables partial application of tuple constructors.
#### TypeApplications
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TypeApplications`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TypeApplications) |
| **Flag** | `-XTypeApplications` |
@ -1123,7 +1158,7 @@ to nice and clean APIs.
#### TypeFamilies
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TypeFamilies`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TypeFamilies) |
| **Flag** | `-XTypeFamilies` |
@ -1150,7 +1185,7 @@ When using Type Families, please keep the following things in mind:
#### TypeFamilyDependencies
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TypeFamilyDependencies`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TypeFamilyDependencies) |
| **Flag** | `-XTypeFamilyDependencies` |
@ -1162,7 +1197,7 @@ would otherwise be ambiguous.
#### TypeOperators
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`TypeOperators`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-TypeOperators) |
| **Flag** | `-XTypeOperators` |
@ -1174,7 +1209,7 @@ expressiveness of type-level APIs.
#### UnicodeSyntax
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`UnicodeSyntax`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-UnicodeSyntax) |
| **Flag** | `-XUnicodeSyntax` |
@ -1183,7 +1218,7 @@ Enables unicode syntax for certain parts of the Haskell language.
#### ViewPatterns
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`ViewPatterns`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-ViewPatterns) |
| **Flag** | `-XViewPatterns` |
@ -1192,6 +1227,7 @@ letting the programmer execute arbitrary logic as part of a pattern match. This
is very useful for the creation of clean APIs.
### Allowed Extensions
These extensions can be used in your code without reservation, but are not
enabled by default because they may interact negatively with other parts of the
codebase.
@ -1199,18 +1235,18 @@ codebase.
#### BlockArguments
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`BlockArguments`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-BlockArguments) |
| **Flag** | `-XBlockArguments` |
Block arguments allow expressions such as `do`, `\`, `if`, `case`, and `let`,
to be used as both arguments to operators and to functions. This can often make
Block arguments allow expressions such as `do`, `\`, `if`, `case`, and `let`, to
be used as both arguments to operators and to functions. This can often make
code more readable than it otherwise would be.
#### GADTs
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------- |
| **Name** | [`GADTs`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-GADTs) |
| **Flag** | `-XGADTs` |
@ -1222,7 +1258,7 @@ available for declaring GADTs.
#### HexFloatLiterals
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`HexFloatLiterals`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-HexFloatLiterals) |
| **Flag** | `-XHexFloatLiterals` |
@ -1232,7 +1268,7 @@ ensure that no rounding or truncation takes place.
#### MagicHash
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`MagicHash`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-MagicHash) |
| **Flag** | `-XMagicHash` |
@ -1243,7 +1279,7 @@ in surface Haskell.
#### NumericUnderscores
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`NumericUnderscores`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-NumericUnderscores) |
| **Flag** | `-XNumericUnderscores` |
@ -1253,7 +1289,7 @@ This extension allows breaking up of long numeric literals using underscores
#### PolyKinds
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`PolyKinds`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-PolyKinds) |
| **Flag** | `-XPolyKinds` |
@ -1268,7 +1304,7 @@ it brings.
#### Quantified Constraints
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`QuantifiedConstraints`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-QuantifiedConstraints) |
| **Flag** | `-XQuantifiedConstraints` |
@ -1280,7 +1316,7 @@ than they would otherwise be able to.
#### RoleAnnotations
| | |
|:---------|:---------------------------------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`RoleAnnotations`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-RoleAnnotations) |
| **Flag** | `-XRoleAnnotations` |
@ -1290,7 +1326,7 @@ specifying the roles of the class and type parameters that they declare.
#### UnboxedSums
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`UnboxedSums`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-UnboxedSums) |
| **Flag** | `-XUnboxedSums` |
@ -1301,7 +1337,7 @@ anonymous sum type, including in pattern matching and at the type level.
#### UnboxedTuples
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`UnboxedTuples`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-UnboxedTuples) |
| **Flag** | `-XUnboxedTuples` |
@ -1310,13 +1346,14 @@ of as a dual to the above `-XunboxedSums` as it allows for the declaration and
manipulation of unboxed product types.
### Allowed With Care
If you make use of any of these extensions in your code, you should accompany
their usage by a source note that explains why they are used.
#### CApiFFI
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------- |
| **Name** | [`CApiFFI`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-CApiFFI) |
| **Flag** | `-XCApiFFI` |
@ -1329,7 +1366,7 @@ against the C language.
#### CPP
| | |
|:---------|:---------------------------------------------------------------------------------------------------------|
| :------- | :------------------------------------------------------------------------------------------------------- |
| **Name** | [`CPP`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-CPP) |
| **Flag** | `-XCPP` |
@ -1341,7 +1378,7 @@ _other_ solution to your problem.
#### PostfixOperators
| | |
|:---------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`PostfixOperators`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-PostfixOperators) |
| **Flag** | `-XPostfixOperators` |
@ -1353,7 +1390,7 @@ the entire codebase.
#### StaticPointers
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`StaticPointers`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-StaticPointers) |
| **Flag** | `-XStaticPointers` |
@ -1365,7 +1402,7 @@ much care must be taken when it is used.
#### UndecidableInstances
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`UndecidableInstances`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-UndecidableInstances) |
| **Flag** | `-XUndecidableInstances` |
@ -1378,16 +1415,17 @@ terminating.
#### UndecidableSuperclasses
| | |
|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------|
| :------- | :----------------------------------------------------------------------------------------------------------------------------------------------- |
| **Name** | [`UndecidableSuperclasses`](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#extension-UndecidableSuperclasses) |
| **Flag** | `-XUndecidableSuperclasses` |
Permits the definition of superclass constraints which can potentially lead to
the non-termination of the type-checker. Much like the above, this is sometimes
necessary but should only be enabled when you are _sure_ that you will not
cause the typechecker to loop.
necessary but should only be enabled when you are _sure_ that you will not cause
the typechecker to loop.
### Disallowed Extensions
If a language extension hasn't been listed in the above sections, then it is
considered to be disallowed throughout the Enso codebases. If you have a good
reason to want to use one of these disallowed extensions, please talk to Ara or

47
docs/style-guide/java.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Java Style Guide
Like many style guides, this Java style guide exists for two primary reasons.
The first is to provide guidelines that result in a consistent code style across
all of the Enso codebases, while the second is to guide people towards a style
@ -34,19 +35,22 @@ programmer burden; there is usually only _one way_ to lay out code correctly.
<!-- /MarkdownTOC -->
## Code Formatting
This section explains the rules for visually laying out your code. They provide
a robust set of guidelines for creating a consistent visual to the code.
Primary code formatting is done using the [Google Java Format](https://github.com/google/google-java-format)
tool, which enforces a clear and consistent style. This is a zero configuration
tool, and hence there is no project-level configuration for this tool. It should
be used for all new Java projects.
Primary code formatting is done using the
[Google Java Format](https://github.com/google/google-java-format) tool, which
enforces a clear and consistent style. This is a zero configuration tool, and
hence there is no project-level configuration for this tool. It should be used
for all new Java projects.
All files must be formatted using this tool before being committed, and this
should be set up as either a precommit hook, or using an integration in your
IDE.
### Naming
Enso has some fairly simple general naming conventions, though the sections
below may provide more rules for use in specific cases.
@ -61,6 +65,7 @@ below may provide more rules for use in specific cases.
- Naming should use American English spelling.
## Package Structure and Naming
Enso follows the
[Java convention for naming packages](https://docs.oracle.com/javase/tutorial/java/package/namingpkgs.html):
package name components may contain only lower case characters and, if
@ -70,11 +75,12 @@ project should be named `org.enso.filemanager`.
When the name of the file in the package is the same as the final component of
the package name, the file should be moved one level up. For examples, if
`File Manager` project contains `FileManager.java` file, then the file should
be placed directly in the `org.enso` package instead of `org.enso.filemanager`.
`File Manager` project contains `FileManager.java` file, then the file should be
placed directly in the `org.enso` package instead of `org.enso.filemanager`.
This is to avoid repetitious constructs like `org.enso.filemanager.FileManager`.
### The Public API
In order to produce as flexible a codebase as possible, we tend not to make use
of access modifiers in our code (`protected`, `private`, and so on). Instead, we
use the concept of `Internal` modules to separate public from private.
@ -85,10 +91,11 @@ you should create a `X.Y.MyType.Internal` package. You can then write the
relevant language construct in that package instead of the source package.
#### Using Access Modifiers
There are, however, a few notable exceptions to the above:
- **Safety:** Privacy modifiers (e.g. `private` and `protected`) should be
used to enforce an API contract around safety.
- **Safety:** Privacy modifiers (e.g. `private` and `protected`) should be used
to enforce an API contract around safety.
- **Reducing Overhead:** As the `Internal` module is a separate module, there
can (under some circumstances) be some overhead for its use. If you are
writing code on a performance-critical path, you may instead make use of
@ -99,6 +106,7 @@ There are, however, a few notable exceptions to the above:
JVM with additional information.
## Commenting
Comments in code are a tricky area to get right as we have found that comments
often expire quickly, and in absence of a way to validate them, remain incorrect
for long periods of time. In order to best deal with this problem, we make the
@ -122,6 +130,7 @@ Code should be written in such a way that it guides you over what it does, and
comments should not be used as a crutch for badly-designed code.
### Documentation Comments
One of the primary forms of comment that we allow across the Enso codebases is
the doc comment. We use these comments to document the public API of a module,
as defined in [The Public API](#the-public-api). For constructs that _are_ part
@ -181,6 +190,7 @@ public class SpanTree<T> implements Tree<T> {
```
### Source Notes
Source Notes is a mechanism for moving detailed design information about a piece
of code out of the code itself. In doing so, it retains the key information
about the design while not impeding the flow of the code. They are used in the
@ -212,9 +222,9 @@ A source note comment is broken into two parts:
already in use.
2. **Source Note:** This is the comment itself, which is a large block comment
placed after the first function in which it is referred to in the module. It
uses the java block-comment syntax `/* ... */`, and the first line names
the note using the same referrer as above: `/* Note [Note Name]`. The name(s)
in the note are underlined using a string of the `~` (tilde) character.
uses the java block-comment syntax `/* ... */`, and the first line names the
note using the same referrer as above: `/* Note [Note Name]`. The name(s) in
the note are underlined using a string of the `~` (tilde) character.
A source note may contain sections within it where necessary. These are titled
using the following syntax: `== Note [Note Name (Section Name)]`, and can be
@ -267,6 +277,7 @@ but can also be used for:
its usage is safe in this context.
### TODO Comments
We follow a simple convention for `TODO` comments in our codebases:
- The line starts with `TODO` or `FIXME`.
@ -284,6 +295,7 @@ For example:
```
### Other Comment Usage
There are, of course, a few other situations where commenting is very useful:
- **Commenting Out:** You may comment out code while developing it, but if you
@ -294,10 +306,12 @@ There are, of course, a few other situations where commenting is very useful:
where the bug has been reported.
## Program Design
Any good style guide goes beyond purely stylistic rules, and also talks about
design styles to use in code.
### Code Complexity
While we often have to write complex functionality, we want to ensure that the
code itself is kept as simple and easy to read as possible. To do this, please
use the following rules:
@ -309,6 +323,7 @@ use the following rules:
comprehension.
### Testing and Benchmarking
New code should always be accompanied by tests. These can be unit, integration,
or some combination of the two, and they should always aim to test the new code
in a rigorous fashion.
@ -334,14 +349,16 @@ _Do not benchmark a development build_ as the data you get will often be
entirely useless.
### Warnings, and Lints
In general, we aim for a codebase that is free of warnings and lints, and we do
this using the following ideas.
#### Warnings
New code should introduce no new warnings onto main. You may build with
warnings on your own branch, but the code that is submitted as part of a PR
should not introduce new warnings. You should also endeavour to fix any warnings
that you come across during development.
New code should introduce no new warnings onto main. You may build with warnings
on your own branch, but the code that is submitted as part of a PR should not
introduce new warnings. You should also endeavour to fix any warnings that you
come across during development.
Sometimes it is impossible to fix a warning (often in situations involving the
use of macros or code-generation). In such cases, you are allowed to suppress

94
docs/style-guide/markdown.md Normal file
View File

@ -0,0 +1,94 @@
---
layout: style-guide
title: Markdown Style Guide
category: style-guide
tags: [style-guide]
order: 5
---
# Markdown Style Guide
Markdown's primary strength is that it's readable in its source form as well as
its rendered form. In the team we seem to have our reading time split evenly
between the two, and so creating a readable format for our markdown
documentation is very important.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
- [Automated Formatting](#automated-formatting)
- [Manual Formatting](#manual-formatting)
- [Title Case](#title-case)
- [Table of Contents](#table-of-contents)
- [In the Future](#in-the-future)
<!-- /MarkdownTOC -->
## Automated Formatting
The bulk of the heavy lifting for formatting our markdown is done by the
formatter [prettier](https://prettier.io). This formatter should be run on the
entire repository using `npx prettier --write .` before every pull request.
For instructions on how to install it, please see our
[contributing guidelines](../CONTRIBUTING.md#getting-set-up-documentation).
If you notice files in generated code being formatted by prettier, please add
them to the [`.prettierignore`](../../.prettierignore) file.
When bumping the version of prettier, please commit the resultant documentation
formatting changes along with the bump as a separate PR from any functional code
changes.
## Manual Formatting
Unfortunately, however, prettier doesn't cope with _all_ of the requirements
that we place on our markdown source files to ensure that they're both easy to
read and easy to navigate.
The following elements require manual formatting.
### Title Case
All section headings should use
[Title Case](https://en.wikipedia.org/wiki/Letter_case#Title_case).
The only exceptions to this rule are where the title refers to the name of an
in-language construct. In these situations, the capitalisation should match that
of the language construct, and it should be surrounded in backticks (`\``).
### Table of Contents
All files should contain a table of contents that shows headings at levels two
and three. This should be placed _directly before_ the first level-two heading.
It should be formatted as follows, with the key elements being:
- Each entry should be a bullet point, with level 2 entries against the baseline
and level 3 entries indented by four spaces.
- The entries should be internal-document links to each of the sections.
- The title of the entry should match the title of the section to which it is
linking.
```md
<!-- MarkdownTOC levels="2,3" autolink="true" -->
- [Level Two](#level-two)
- [Level Two Again](#level-two-again)
- [Level Three](#level-three)
<!-- /MarkdownTOC -->
```
These table of contents can be automatically generated by plugins to common
editors.
A document needing its table of contents to show level four entries is usually
indicative of portions of the document needing to be extracted. If you feel that
you have a situation where this is not the case, please talk to a documentation
code owner.
## In the Future
As prettier doesn't enforce _all_ of our markdown style guidelines, in the
future a new formatter will be written that will enforce all of our guidelines
in one go.

66
docs/style-guide/rust.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Rust Style Guide
Like many style guides, this Rust style guide exists for two primary reasons.
The first is to provide guidelines that result in a consistent code style across
all of the Enso codebases, while the second is to guide people towards a style
@ -46,6 +47,7 @@ programmer burden; there is usually only _one way_ to lay out code correctly.
<!-- /MarkdownTOC -->
## Code Formatting
This section explains the rules for visually laying out your code. They provide
a robust set of guidelines for creating a consistent visual to the code.
@ -69,10 +71,12 @@ described below, but even so, many portions of this guide will need to be
handled manually.
### Line Width
Each line in the source file should be of a maximum of 100 characters of text.
This includes comments.
### Imports
The imports section at the top of a file should be separated into four groups.
These groups should be sorted in alphabetical order and are divided as follows:
@ -100,6 +104,7 @@ use nalgebra::Vector3;
```
### Sections
Rust source files should be divided into sections, with a header placed before
the definition of each new concept in a file.
@ -125,6 +130,7 @@ for a sub-section is as follows.
At least one section should be defined in every file.
#### An Example of Using Sections
Here is a large-scale example of how sections should be used in source files.
```rust
@ -222,6 +228,7 @@ impl<OnChange:Callback0> HierarchicalTransform<OnChange> {
```
### Vertical Spacing
We use the following rules for the amount of vertical space separating various
constructs in the source:
@ -237,6 +244,7 @@ take the maximum of the spacings that apply. For example, if you have a section
following the imports, you only use three lines of spacing.
### Multi-Line Expressions
In an ideal world, all expressions in the code should be a single line. This is
because multi-line expressions are usually hard to read, and because they can
introduce lots of noise in the code. In the vast majority of cases, the presence
@ -246,6 +254,7 @@ Please try to refactor portions of multi-line expressions to well-named
variables, and divide them up to a set of single-line expressions.
#### Multi-Line Expression Examples
The following is an example of poorly formatted code:
```rust
@ -271,6 +280,7 @@ pub fn new() -> Self {
```
### Vertical Alignment
In order to create a visual flow to our code that aids readability, the
following constructs should be aligned vertically where possible:
@ -280,6 +290,7 @@ following constructs should be aligned vertically where possible:
- Similar parameters or types
#### A Vertical Alignment Example
The following is an example of a function that correctly uses the vertical
alignment rules above:
@ -297,6 +308,7 @@ impl Printer for GlobalVarStorage {
```
### Spacing
The following spacing rules are _also_ employed in order to create a visual flow
to our code to aid readability:
@ -307,6 +319,7 @@ to our code to aid readability:
- Operators are always spaced: `let foo = a + b * c;`
#### Spacing Examples as Function Definitions
The following function definitions are all good examples of correct use of
spacing.
@ -370,11 +383,11 @@ where D:AsRef<str>
```
### Impl Definitions
In order to aid in fast discovery of the header of an impl definition, we use
the following style. In all cases, the `where` block should be placed after a
line break.
```rust
// No constraints
impl<T> Printer for Option<T> {
@ -409,6 +422,7 @@ We also have a specific _ordering_ for `impl` definitions. It is as follows:
Each of these should be accompanied by a sub-heading.
### Getters and Setters
We have the following rules for getters and setters in our codebase.
- Getters do not have the `get_` prefix, while setters do have the `set_`
@ -436,6 +450,7 @@ Getters and setters should be implemented in separate `impl`, blocks, each with
their own subheading.
### Trait Exports
All names should be designed to be used in a qualified fashion. This does,
however, make one situation quite tricky. In order to use methods defined inside
a trait, that trait has to be in scope.
@ -463,6 +478,7 @@ Once we have such a definition, we can import traits into scope using the simple
unnecessary qualification.
## Naming
Enso has some fairly simple general naming conventions, though the sections
below may provide more rules for use in specific cases.
@ -473,9 +489,8 @@ below may provide more rules for use in specific cases.
- Short variable names such as `a` and `b` should only be used in the following
contexts:
- Where there is no other appropriate name.
- Named lifetimes.
They should _never_ be used to refer to temporary data in a function, as all
temporaries should be given descriptive names.
- Named lifetimes. They should _never_ be used to refer to temporary data in a
function, as all temporaries should be given descriptive names.
- Names should be descriptive, even if this makes them longer.
- Any function that performs an unsafe operation that is not documented in its
type (e.g. `fn head<T>(ts: Vec<T>) -> T`, which fails if the list is empty),
@ -484,8 +499,9 @@ below may provide more rules for use in specific cases.
- Naming should use American English spelling.
## Package Structure and Naming
Enso follows the standard rust convention for structuring crates, as provided
by `cargo new`. This is discussed more in depth
Enso follows the standard rust convention for structuring crates, as provided by
`cargo new`. This is discussed more in depth
[here](https://learning-rust.github.io/docs/a4.cargo,crates_and_basic_project_structure.html#Project-Structure).
In order to match up with the project naming convention we use for Scala and
@ -493,6 +509,7 @@ Java projects, any rust code must be in a directory named using `UpperCamelCase`
in the root of the project (e.g. `enso/BaseGL`).
### The Public API
Whereas Rust defaults to making module members _private_ by default, this is not
the philosophy used by the Enso codebases. We tend to want our codebase to be
flexible for consumers, so we tend to avoid making things private. Instead, we
@ -504,6 +521,7 @@ you should create a `foo.bar.baz.internal` package. You can then write the
relevant language construct in that package instead of the source package.
#### Using Access Modifiers
Given Rust's performance guarantees, making things `pub` has no impact on the
performance of the compiled code. As a result, the _only_ circumstance under
which things are allowed to not be `pub` is when doing so would allow consumers
@ -511,9 +529,12 @@ of an API to break internal guarantees provided by that API (e.g. building an
immutable collection on top of a mutable buffer).
## Build Tooling
All Rust projects are built and managed using [cargo](https://doc.rust-lang.org/cargo/).
All Rust projects are built and managed using
[cargo](https://doc.rust-lang.org/cargo/).
## Commenting
Comments in code are a tricky area to get right as we have found that comments
often expire quickly, and in absence of a way to validate them, remain incorrect
for long periods of time. In order to best deal with this problem, we make the
@ -537,6 +558,7 @@ Code should be written in such a way that it guides you over what it does, and
comments should not be used as a crutch for badly-designed code.
### Documentation Comments
One of the primary forms of comment that we allow across the Enso codebases is
the doc comment. We use these comments to document the public API of a module,
as defined in [The Public API](#the-public-api). For constructs that _are_ part
@ -548,8 +570,9 @@ of the public API, the following should be documented:
for how to use that function.
Documentation comments are intended for consumption by the users of the API, and
are written using the standard [rustdoc](https://doc.rust-lang.org/rustdoc/index.html)
syntax. Doc comments should contain:
are written using the standard
[rustdoc](https://doc.rust-lang.org/rustdoc/index.html) syntax. Doc comments
should contain:
1. **Summary:** A one-line summary of the construct's behaviour or purpose.
2. **Description (Optional):** Any useful information that would be necessary
@ -587,6 +610,7 @@ You may document _more_ than what is specified here, but this is the _minimum_
required for acceptance at code-review time.
### Source Notes
Source Notes is a mechanism for moving detailed design information about a piece
of code out of the code itself. In doing so, it retains the key information
about the design while not impeding the flow of the code. They are used in the
@ -658,6 +682,7 @@ pub trait Tree<T> {
```
### TODO Comments
We follow a simple convention for `TODO` comments in our codebases:
- The line starts with `TODO` or `FIXME`.
@ -673,6 +698,7 @@ For example:
```
### Other Comment Usage
There are, of course, a few other situations where commenting is very useful:
- **Commenting Out:** You may comment out code while developing it, but if you
@ -683,10 +709,12 @@ There are, of course, a few other situations where commenting is very useful:
where the bug has been reported.
## Program Design
Any good style guide goes beyond purely stylistic rules, and also talks about
design styles to use in code.
### Code Complexity
While we often have to write complex functionality, we want to ensure that the
code itself is kept as simple and easy to read as possible. To do this, please
use the following rules:
@ -698,6 +726,7 @@ use the following rules:
comprehension.
### Safety
Whereas most languages don't have a concept of _safety_, rust comes with a built
in notion of `unsafe`. When working with `unsafe` functions and code blocks, you
must account for the following:
@ -713,12 +742,14 @@ must account for the following:
Furthermore, we do not allow for code containing pattern matches that can fail.
### Testing and Benchmarking
New code should always be accompanied by tests. These can be unit, integration,
or some combination of the two, and they should always aim to test the new code
in a rigorous fashion.
- Testing should be performed as described in [the Rust book](https://doc.rust-lang.org/book/ch11-00-testing.html)
and should use the functionality for testing built into the language.
- Testing should be performed as described in
[the Rust book](https://doc.rust-lang.org/book/ch11-00-testing.html) and
should use the functionality for testing built into the language.
- Tests should cover as much code as possible, and may be a combination of unit
and integration tests.
@ -727,7 +758,8 @@ These are intended to allow us to catch performance regressions as the code
evolves, but also ensure that we have some idea of the code's performance in
general.
- We use nightly rust in order to access the built-in [benchmarking](https://doc.rust-lang.org/unstable-book/library-features/test.html)
- We use nightly rust in order to access the built-in
[benchmarking](https://doc.rust-lang.org/unstable-book/library-features/test.html)
functionality.
- We measure time, CPU, and memory usage where possible.
- Where relevant, benchmarks may set thresholds which, when surpassed, cause the
@ -738,14 +770,16 @@ _Do not benchmark a development build_ as the data you get will often be
entirely useless.
### Warnings, and Lints
In general, we aim for a codebase that is free of warnings and lints, and we do
this using the following ideas:
#### Warnings
New code should introduce no new warnings onto main. You may build with
warnings on your own branch, but the code that is submitted as part of a PR
should not introduce new warnings. You should also endeavour to fix any warnings
that you come across during development.
New code should introduce no new warnings onto main. You may build with warnings
on your own branch, but the code that is submitted as part of a PR should not
introduce new warnings. You should also endeavour to fix any warnings that you
come across during development.
Sometimes it is impossible to fix a warning (often in situations involving the
use of macros). In such cases, you are allowed to suppress the warning locally,

37
docs/style-guide/scala.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Scala Style Guide
Like many style guides, this Scala style guide exists for two primary reasons.
The first is to provide guidelines that result in a consistent code style across
all of the Enso codebases, while the second is to guide people towards a style
@ -36,6 +37,7 @@ programmer burden; there is usually only _one way_ to lay out code correctly.
<!-- /MarkdownTOC -->
## Code Formatting
This section explains the rules for visually laying out your code. They provide
a robust set of guidelines for creating a consistent visual to the code.
@ -53,6 +55,7 @@ installed, and be using IntelliJ 2019.1 or later. You should _not_ use the
independent Scalafmt plugin.
## Naming
Enso has some fairly simple general naming conventions, though the sections
below may provide more rules for use in specific cases.
@ -73,6 +76,7 @@ below may provide more rules for use in specific cases.
- Naming should use American English spelling.
## Package Structure and Naming
Enso follows the
[Java convention for naming packages](https://docs.oracle.com/javase/tutorial/java/package/namingpkgs.html):
package name components may contain only lower case characters and, if
@ -87,6 +91,7 @@ be placed directly in the `org.enso` package instead of `org.enso.filemanager`.
This is to avoid repetitious constructs like `org.enso.filemanager.FileManager`.
### The Public API
In order to produce as flexible a codebase as possible, we tend not to make use
of access modifiers in our code (`public`, `private`, and so on). Instead, we
use the concept of `Internal` modules to separate public from private.
@ -97,6 +102,7 @@ you should create a `X.Y.MyType.Internal` package. You can then write the
relevant language construct in that package instead of the source package.
#### Using Access Modifiers
There are, however, a few notable exceptions to the above:
- **Safety:** Privacy modifiers (e.g. `private` and `private[this]`) should be
@ -113,6 +119,7 @@ There are, however, a few notable exceptions to the above:
JVM with additional information.
## Build Tooling
All Scala projects in the Enso organisation should manage their dependencies and
build setup using [SBT](hhttps://www.scala-sbt.org/1.x/docs/index.html).
@ -120,6 +127,7 @@ If you are using IntelliJ, please ensure that you select to use the SBT shell
for both imports and builds.
## Commenting
Comments in code are a tricky area to get right as we have found that comments
often expire quickly, and in absence of a way to validate them, remain incorrect
for long periods of time. In order to best deal with this problem, we make the
@ -143,6 +151,7 @@ Code should be written in such a way that it guides you over what it does, and
comments should not be used as a crutch for badly-designed code.
### Documentation Comments
One of the primary forms of comment that we allow across the Enso codebases is
the doc comment. We use these comments to document the public API of a module,
as defined in [The Public API](#the-public-api). For constructs that _are_ part
@ -189,8 +198,9 @@ object Actions {
```
Documentation comments are intended for consumption by the users of the API, and
are written using the standard [scaladoc](https://docs.scala-lang.org/style/scaladoc.html)
syntax. Doc comments should contain:
are written using the standard
[scaladoc](https://docs.scala-lang.org/style/scaladoc.html) syntax. Doc comments
should contain:
1. **Summary:** A one-line summary of the construct's behaviour. This should be
a valid sentence with 'this X' (where X = function, trait, etc) prepended to
@ -242,6 +252,7 @@ You may document _more_ than what is specified here, but this is the _minimum_
required for acceptance at code-review time.
### Source Notes
Source Notes is a mechanism for moving detailed design information about a piece
of code out of the code itself. In doing so, it retains the key information
about the design while not impeding the flow of the code. They are used in the
@ -273,9 +284,9 @@ A source note comment is broken into two parts:
already in use.
2. **Source Note:** This is the comment itself, which is a large block comment
placed after the first function in which it is referred to in the module. It
uses the scala block-comment syntax `/* ... */`, and the first line names
the note using the same referrer as above: `/* Note [Note Name]`. The name(s)
in the note are underlined using a string of the `~` (tilde) character.
uses the scala block-comment syntax `/* ... */`, and the first line names the
note using the same referrer as above: `/* Note [Note Name]`. The name(s) in
the note are underlined using a string of the `~` (tilde) character.
A source note may contain sections within it where necessary. These are titled
using the following syntax: `== Note [Note Name (Section Name)]`, and can be
@ -319,6 +330,7 @@ def prepRHS (env : SimplEnv, outExpr : OutExpr) : SimplM[SimplEnv, OutExpr] = {
```
### TODO Comments
We follow a simple convention for `TODO` comments in our codebases:
- The line starts with `TODO` or `FIXME`.
@ -336,6 +348,7 @@ For example:
```
### Other Comment Usage
There are, of course, a few other situations where commenting is very useful:
- **Commenting Out:** You may comment out code while developing it, but if you
@ -346,10 +359,12 @@ There are, of course, a few other situations where commenting is very useful:
where the bug has been reported.
## Program Design
Any good style guide goes beyond purely stylistic rules, and also talks about
design styles to use in code.
### Code Complexity
While we often have to write complex functionality, we want to ensure that the
code itself is kept as simple and easy to read as possible. To do this, please
use the following rules:
@ -361,6 +376,7 @@ use the following rules:
comprehension.
### Safety
It is incredibly important that we can trust the code that we use, and hence we
tend to disallow the definition of unsafe functions in our public API. When
defining an unsafe function, you must account for the following:
@ -376,6 +392,7 @@ defining an unsafe function, you must account for the following:
Furthermore, we do not allow for code containing pattern matches that can fail.
### Testing and Benchmarking
New code should always be accompanied by tests. These can be unit, integration,
or some combination of the two, and they should always aim to test the new code
in a rigorous fashion.
@ -401,14 +418,16 @@ _Do not benchmark a development build_ as the data you get will often be
entirely useless.
### Warnings, and Lints
In general, we aim for a codebase that is free of warnings and lints, and we do
this using the following ideas:
#### Warnings
New code should introduce no new warnings onto main. You may build with
warnings on your own branch, but the code that is submitted as part of a PR
should not introduce new warnings. You should also endeavour to fix any warnings
that you come across during development.
New code should introduce no new warnings onto main. You may build with warnings
on your own branch, but the code that is submitted as part of a PR should not
introduce new warnings. You should also endeavour to fix any warnings that you
come across during development.
Sometimes it is impossible to fix a warning (often in situations involving the
use of macros). In such cases, you are allowed to suppress the warning locally,

34
docs/style-guide/yaml.md Normal file
View File

@ -0,0 +1,34 @@
---
layout: style-guide
title: YAML Style Guide
category: style-guide
tags: [style-guide]
order: 6
---
# YAML Style Guide
In order to make our YAML configuration easy to read, we use an automated
formatter to ensure that it's up to scratch.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
- [Automated Formatting](#automated-formatting)
<!-- /MarkdownTOC -->
## Automated Formatting
The bulk of the heavy lifting for formatting our YAML is done by the formatter
[prettier](https://prettier.io). This formatter should be run on the entire
repository using `npx prettier --write .` before every pull request.
For instructions on how to install it, please see our
[contributing guidelines](../CONTRIBUTING.md#getting-set-up-documentation).
If you notice files in generated code being formatted by prettier, please add
them to the [`.prettierignore`](../../.prettierignore) file.
When bumping the version of prettier, please commit the resultant configuration
formatting changes along with the bump as a separate PR from any functional code
changes.

3
docs/syntax/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso's Syntax
When working with a programming language, the syntax is the first thing that a
user encounters. This makes it _utterly integral_ to how users experience the
language, and, in the case of Enso, the tool as a whole.
@ -29,7 +30,7 @@ The various components of Enso's syntax are described below:
- [**Encoding:**](./encoding.md) The source encoding of Enso files.
- [**Naming:**](./naming.md) The naming of Enso language constructs.
- [**Layout Rules:**](./layout.md) The layout rules for Enso code.
- [**Imports and Exports:**](./imports.md) The syntax and rules for importing
- [**Imports and Exports:**](./imports.md) The syntax and rules for importing
modules and exporting module functions.
- [**Literals:**](./literals.md) The syntax for Enso literals.
- [**Assignment:**](./assignment.md) The syntax for various forms of assignment

20
docs/syntax/assignment.md
View File

@ -7,6 +7,7 @@ order: 6
---
# Assignment Expressions
Assignment syntax in Enso is fairly magical, given that it is the language's
syntax for monadic bind.
@ -23,6 +24,7 @@ syntax for monadic bind.
<!-- /MarkdownTOC -->
## How Assignment Works
Assignment in Enso operates as follows:
- An assignment is an _expression_.
@ -40,10 +42,11 @@ functions, extension methods, and to perform pattern matching. Each different
case will see an appropriate desugaring applied (see below).
Please note that not _all_ occurrences of the `=` operator are assignments in
the general sense. The above rules do not apply when using said operator to
pass arguments by name.
the general sense. The above rules do not apply when using said operator to pass
arguments by name.
## Function Definitions
If the left hand side of an assignment is syntactically a prefix application
chain, where the left-most name is a _variable_ name, the assignment is
considered to introduce a _function definition_ (the syntax sugared version).
@ -65,6 +68,7 @@ Please note that by the rules of naming specified previously, if an operator
occurs in the same position as `a` it will also be defined.
## Pattern Match Bindings
If the left hand side of an assignment is syntactically a prefix application
chain, where the left-most name is a _type_ name, the assignment is considered
to introduce a pattern match binding.
@ -91,6 +95,7 @@ This also works for operators in an infix position, where its operands will be
matched against.
## Extension Methods
There are two cases where an assignment creates an extension method:
1. **Method Syntax:** If the left-hand-side of an assignment is syntactically a
@ -98,12 +103,13 @@ There are two cases where an assignment creates an extension method:
application of `.`, this assignment is considered to introduce an extension
method.
2. **Function Syntax:** If the left hand side of an assignment is syntactically
a prefix application chain where the left-most expression is a variable
identifier and the second expression from the left is a variable named `this`
with an explicit type ascription, this is also considered to introduce an
extension method.
a prefix application chain where the left-most expression is a variable
identifier and the second expression from the left is a variable named `this`
with an explicit type ascription, this is also considered to introduce an
extension method.
### Method Syntax
This syntax for extension methods works as follows:
- The target of the method syntax (left argument to `.`) defines the type on
@ -117,6 +123,7 @@ My_Type.method_name a b c = ...
```
### Function Syntax
This syntax for extension methods works as follows:
- The `this` argument type is used to define the type on which the extension
@ -128,6 +135,7 @@ method_name (this : My_Type) a b c = ...
```
## Top-Level Assignments
In order to aid with disambiguation, any binding made in the root scope without
an explicit target is implicitly defined on a type representing the current
module. For example, a binding `main = ...` is implicitly `here.main = ...`.

14
docs/syntax/comments.md
View File

@ -7,6 +7,7 @@ order: 13
---
# Comments
Enso supports a variety of types of comments:
- **Disable Comments:** TODO
@ -33,6 +34,7 @@ Enso supports a variety of types of comments:
<!-- /MarkdownTOC -->
## Disable Comments
Disable comments are the standard form of comment seen in a programming language
in that they prevent a given piece of code from executing. In Enso, they are
created by prefixing the expression to disable with the `#` character.
@ -46,6 +48,7 @@ These aren't _exactly_ like most language's disable comments however:
in the visual syntax.
## Documentation Comments
Documentation comments allow users to attach documentation to Enso language
constructs that can later be displayed in a rich format for users of the API.
Such comments are automatically connected to the language construct, and can be
@ -70,6 +73,7 @@ mistakes will be highlighted to the user.
The documentation syntax is broken down into the following elements.
### Tags
Tags allow users to annotate their construct with information about its usage
state. The documentation syntax supports the following tags:
@ -95,6 +99,7 @@ If the user provides an unknown tag the documentation will contain that tag, but
it will be undefined.
### Sections
Documentation comments can be broken up into sections, with each section
delineated by significant whitespace.
@ -123,6 +128,7 @@ different types of section:
section, prefix the title with `>`.
### Links
Users are able to embed links and images into their documentation. These links
can serve to provide access to external resources or demonstrations, and also
link between various program constructs.
@ -138,6 +144,7 @@ displayed like standard hyperlinks.
> - We probably want a construct that lets you reference other API constructs.
### Lists
The Enso documentation syntax also supports ordered and unordered lists. These
can be nested, and the nesting may swap the types. Both list types must be
intended some multiple of 2 spaces from the left margin of the documentation
@ -150,6 +157,7 @@ To nest a list inside another list, add another 2-character indent to the nested
list.
### Code
The Enso documentation syntax allows users to write code that will be displayed
as code rather than prose. It supports two types of code.
@ -158,8 +166,9 @@ as code rather than prose. It supports two types of code.
section will be formatted as a code block.
### Text Formatting
Enso's documentation syntax also supports some basic syntax for adding rich
text formatting to the documentation.
Enso's documentation syntax also supports some basic syntax for adding rich text
formatting to the documentation.
- **Italics:** Enclosing text in `_` (e.g. `_Italics_`).
- **Bold:** Enclosing text in `*` (e.g. `*Bold*`).
@ -168,4 +177,3 @@ text formatting to the documentation.
These syntaxes may be combined, and the order of opening need not equal the
order of closing. However, if the formatting syntaxes are not closed, this will
result in an error.

3
docs/syntax/encoding.md
View File

@ -7,6 +7,7 @@ order: 1
---
# Source File Encoding
While many modern programming languages are moving in a direction of being
liberal with the input they accept, we find that this often leads to the
resultant code being more difficult to use.
@ -21,9 +22,11 @@ This file describes the source file encoding for Enso files.
<!-- /MarkdownTOC -->
## Source Encoding
All input source in Enso is UTF-8 encoded.
## Indentation
Indentation in Enso is performed using spaces. An indent level is 4 spaces.
This is intentionally not made configurable.

8
docs/syntax/function-arguments.md
View File

@ -7,6 +7,7 @@ order: 11
---
# Function Arguments
One of the biggest usability innovations of Enso is the set of argument types
that it supports. The combination of named and defaulted arguments with a
curried language creates a tool in which it is very clear to express even
@ -25,11 +26,13 @@ complex APIs.
<!-- /MarkdownTOC -->
## Positional Arguments
Much like most programming languages, functions in Enso can be called with their
arguments provided positionally. This is the simple case that everybody is
familiar with.
## Named Arguments
All arguments in Enso are defined with a name. Like all programming languages,
this is necessary for that argument to be used. However, what Enso allows is for
users to then _call_ those arguments by name.
@ -48,6 +51,7 @@ This is a great usability boon as in complex APIs it can often be difficult to
remember the order or arguments.
## Defaulted Arguments
Enso also allows users to define their functions with _defaults_ for the
function's arguments. This is very useful for complex APIs as it allows users to
experiment and iterate quickly by only providing the arguments that they want to
@ -63,6 +67,7 @@ customise.
arguments for the purposes of currying.
## Optional Arguments
There are certain cases where the type information for an argument may be able
to be inferred by the compiler. This is best explained by example. Consider the
implementation of a `read` function that reads text and outputs a value of a
@ -102,6 +107,7 @@ read text this? = t.fromText text
```
## Splats Arguments (Variadics)
Enso provides users with the ability to define variadic functions, or _splats_
functions in our terminology. These are very useful for defining expressive APIs
and flexible code.
@ -116,6 +122,7 @@ and flexible code.
> - Do we even want this?
## Type Applications
There are sometimes cases where the user wants to explicitly refine the type of
an argument at the _call_ site of a function. This can be useful for debugging,
and for writing ad-hoc code. Much like the named-arguments in applications
@ -127,6 +134,7 @@ above, Enso also provides a syntax for refining types at the application site.
subtype for the type inferred for (or defined for) the function being called.
## Underscore Arguments
Enso provides the `_` argument as a quick way to create a lambda from a function
call. It obeys the following rules.

60
docs/syntax/functions.md
View File

@ -7,6 +7,7 @@ order: 10
---
# Defining Functions
Enso is a purely-functional programming language. As a result it has support for
[first-class and higher-order functions](https://en.wikipedia.org/wiki/Functional_programming#First-class_and_higher-order_functions),
meaning that you can pass functions as arguments to other functions, return
@ -32,13 +33,14 @@ arguments.
<!-- /MarkdownTOC -->
## Lambdas
The most primitive non-atom construct in Enso is the lambda. This is an
anonymous function in one argument. A lambda is defined using the `->` operator,
where the left hand side is an argument, and the right hand side is the body of
the function (containing arbitrary code).
Some functional languages such as Haskell allow for the definition of a lambda
with multiple arguments, but in Enso the type signature use of `-> `and the
with multiple arguments, but in Enso the type signature use of `->`and the
lambda use of `->` are one and the same. We do not want to have to put the
components of a type signature in parentheses, so we only allow one argument
before each arrow.
@ -68,6 +70,7 @@ reference rule and write (in this case) `a -> A -> a`.
> - Do we want any automated unification to take place in the shadowing case?
## Defining Functions
A function definition is just syntactic sugar for the definition of a lambda,
and hence has all the properties that a lambda does. Syntactically, functions
are defined in a similar way to variables. The only difference is that the
@ -95,6 +98,7 @@ binds the function name. This means that:
still be inferred.
## Methods
Enso makes a distinction between functions and methods. In Enso, a method is a
function where the first argument (known as the `this` argument) is associated
with a given atom. Methods are dispatched dynamically based on the type of the
@ -106,39 +110,40 @@ Methods can be defined in Enso in two ways:
definition is automatically converted to a method on all the atoms defined in
the body of that type definition.
```ruby
type Maybe a
Nothing
type Just (value : a)
```ruby
type Maybe a
Nothing
type Just (value : a)
isJust = case this of
Nothing -> False
Just _ -> True
```
isJust = case this of
Nothing -> False
Just _ -> True
```
2. **As an Extension Method:** A function defined _explicitly_ on an atom counts
as an extension method on that atom. It can be defined on a typeset to apply
to all the atoms within that typeset.
```ruby
Number.floor = case this of
Integer -> ...
...
```
```ruby
Number.floor = case this of
Integer -> ...
...
```
3. **As a Function with an Explicit `this` Argument:** A function defined with
the type of the `this` argument specified to be a type.
```ruby
floor (this : Number) = case this of
Integer -> ...
```
```ruby
floor (this : Number) = case this of
Integer -> ...
```
If the user does not explicitly specify the `this` argument by name when
defining a method (e.g. they use the `Type.name` syntax), it is implicitly added
to the start of the argument list.
## Universal Call Syntax
Calling a function or method is, in general, as simple as applying it to some
arguments. However, as Enso supports both methods and functions, it is very
important that users do not have to think about which of the two they are using
@ -156,6 +161,7 @@ The rules for the uniform syntax call translation in Enso are as follows:
2. For an expression `fn t <args>`, this is equivalent to `t.fn <args>`.
## Code Blocks
Top-level blocks in the language are evaluated immediately. This means that the
layout of the code has no impact on semantics of the code:
@ -225,6 +231,7 @@ foo =
```
## Operators
In Enso, an operator is a function with a non-alphanumeric name (e.g. `+`). We
only support binary operators, with left and right arguments.
@ -252,10 +259,11 @@ parts:
```
### Precedence
Operator precedence in Enso is a collection of rules that reflect conventions
about which operations to perform first in order to evaluate a given expression
that contains operators. However, operator precedence in Enso differs from
many other programming languages.
that contains operators. However, operator precedence in Enso differs from many
other programming languages.
- Precedence is not set at fixed levels, but is instead defined in relation to
the precedence of other operators.
@ -266,9 +274,9 @@ many other programming languages.
which retains the same precedence level regardless of whether it is surrounded
by spaces or not.
This space-based precedence may seem strange coming from other languages, but
it allows for writing _far_ cleaner code than other functional languages. This
is best demonstrated by example. Consider the following code:
This space-based precedence may seem strange coming from other languages, but it
allows for writing _far_ cleaner code than other functional languages. This is
best demonstrated by example. Consider the following code:
```ruby
list = 1 .. 100
@ -295,6 +303,7 @@ result = 1..100 . each random . take 10 . sort
```
### Sections
An operator section is a nice shorthand for partially applying an operator. It
works as follows.
@ -307,6 +316,7 @@ works as follows.
sections.
## Mixfix Functions
A mixfix function is a function that is made up of multiple sections. They are
defined using a special syntax, and operate as follows:
@ -317,8 +327,8 @@ defined using a special syntax, and operate as follows:
separate operator, allowing you to write an indented block of code after each
section.
Probably the best-known example of a mixfix function is `if-then-else`, which
is indeed defined in the Enso standard library.
Probably the best-known example of a mixfix function is `if-then-else`, which is
indeed defined in the Enso standard library.
```ruby
if foo == bar then frob else

18
docs/syntax/imports.md
View File

@ -7,20 +7,22 @@ order: 4
---
# Imports and Exports
In order to properly modularise and work with Enso code, the language provides
a robust mechanism for importing code from modules, and also re-exporting that
In order to properly modularise and work with Enso code, the language provides a
robust mechanism for importing code from modules, and also re-exporting that
code from modules.
<!-- MarkdownTOC levels="2,3" autolink="true" -->
- [Import Syntax](#import-syntax)
- [Visibility of Imported Bindings](#visibility-of-imported-bindings)
- [Visibility of Imported Bindings](#visibility-of-imported-bindings)
- [Export Syntax](#export-syntax)
- [Visibility of Export Bindings](#visibility-of-export-bindings)
- [Visibility of Export Bindings](#visibility-of-export-bindings)
<!-- /MarkdownTOC -->
## Import Syntax
Importing a module is a way to bring its contents into scope in the current
module. Imports in Enso appear as follows:
@ -37,8 +39,8 @@ From there, Enso imports are broken up into four main categories:
module name. It consists of the `import` keyword followed by a module path
followed by the `as` keyword, followed by a referent name.
3. **Restricted Imports:** These import only the specific symbols from the
module into the current scope. They consist of the `import` keyword, followed
by a module path, followed by the `only` keyword, followed by a
module into the current scope. They consist of the `import` keyword,
followed by a module path, followed by the `only` keyword, followed by a
space-separated list of symbols.
4. **Hiding Imports:** These are the inverse of restricted imports, and import
_all_ symbols other than the named ones into the current scope. They consist
@ -58,16 +60,19 @@ import D hiding symbol_1 symbol_2 # hiding
import E as U only symbol_3 # qualified + restricted
import F as V hiding symbol_4 # qualified + hiding
```
Imports in Enso _may_ introduce ambiguous symbols, but this is not an error
until one of the ambiguous symbols is _used_ in Enso code.
### Visibility of Imported Bindings
When importing a module `X` into the current module `Y`, the bindings from `X`
made available by the import (see above) become available in `Y`. However, Enso
does not re-export imported bindings from a module by default, so the imported
bindings from `X` are not visible in a module _importing_ `Y`.
## Export Syntax
In order to allow for easy composition and aggregation of code, Enso provides
its users with a mechanism to _export_ imported elements from modules. They
appear in Enso as follows:
@ -114,5 +119,6 @@ being exported into the module declaring the export. This means that exports
that create name clashes need to be resolved at the _export_ site.
### Visibility of Export Bindings
Bindings exported from a module `X` are available in an identical fashion to
bindings that are _defined_ in the module `X`.

7
docs/syntax/layout.md
View File

@ -7,6 +7,7 @@ order: 3
---
# Layout Rules
Enso is a layout-aware programming language, in that layout rules are used to
determine code structure. The layout rules in Enso are intended to provide for
an intuitive way to format code.
@ -25,6 +26,7 @@ This document describes the layout rules for Enso's white-space.
<!-- /MarkdownTOC -->
## Maximum Line Length
The maximum length of a line in an Enso source file is restricted to 80
characters outside of text blocks. If your code exceeds this limit, the compiler
will emit a warning message.
@ -38,6 +40,7 @@ in user code. The reasoning behind this is as follows:
views.
## Indented Blocks
Indentation in Enso is used to start a block. Every indented line is considered
to be a sub-structure of the nearest previous line with lower indentation. We
refer to these as the 'child' and the 'parent' lines respectively. This means
@ -54,6 +57,7 @@ In addition, we have a set of custom layout rules that impact exactly how blocks
are defined. These are described in the following subsections.
### Trailing Operator on the Parent Line
If a line ends with an operator then all of its child lines form a
[_code_ block](./functions.md/#code-blocks). The most common usage of this kind
of indented block is a function definition body (following the `=` or `->`).
@ -64,6 +68,7 @@ test = a -> b ->
```
### Leading Operator on All Child Lines
If all the child lines in a block begin with an operator, the lines in the block
are considered to form a single expression.
@ -85,6 +90,7 @@ nums = 1..100
```
### No Leading or Trailing Operators
In the case where neither the parent line ends with a trailing operator, or the
child lines begin with an operator, every child line is considered to form a
separate expression passed as an argument to the parent expression. The most
@ -99,6 +105,7 @@ geo2 = sphere
```
### Debug Line Breaks
In certain cases it may be useful to debug line breaks in code. To this end, we
provide a debug line-break operator `\\` which, when placed at the beginning of
a child line tells Enso to glue that line to the end of the previous one.

6
docs/syntax/literals.md
View File

@ -7,6 +7,7 @@ order: 5
---
# Literals
Enso supports a small set of literals that allow the expression of some common
types in literal form in the source code.
@ -21,6 +22,7 @@ types in literal form in the source code.
<!-- /MarkdownTOC -->
## Numeric Literals
Enso provides rich support for numeric literals, including literals that use
different numeric bases. It does, of course, support floating point numerals as
well.
@ -58,6 +60,7 @@ binary = 2_10011101010
> literals, or whether these should be relegated to type constructors.
## Text Literals
Enso provides rich support for textual literals in the language, supporting both
raw and interpolated strings natively.
@ -79,6 +82,7 @@ raw and interpolated strings natively.
```
### Inline Text Literals
In Enso, inline text literals are opened and closed using the corresponding
quote type for the literal. They may contain escape sequences but may _not_ be
broken across lines.
@ -89,6 +93,7 @@ inline_interpolated = 'Foo `bar` baz'
```
### Text Block Literals
In Enso, text block literals rely on _layout_ to determine the end of the block,
allowing users to only _open_ the literal. Block literals are opened with three
of the relevant quote type, and the contents of the block are determined by the
@ -112,6 +117,7 @@ not_string_expr = foo bar
```
## Vector Literals
Enso also supports vector literals, which allow users to create literal vectors
of elements.

15
docs/syntax/macros.md
View File

@ -7,6 +7,7 @@ order: 8
---
# The Enso Macro Syntax
Enso provides a macro system that allows users to perform AST to AST
transformations on the provided pieces of code. While many languages' macros
provide their users with access to the compilation and type-checking phases
@ -32,11 +33,12 @@ provide their users with access to the compilation and type-checking phases
<!-- /MarkdownTOC -->
## Annotations
Much like annotations on the JVM, annotations in Enso are tags that perform a
purely syntactic transformation on the entity to which they are applied. The
implementation of this requires both parser changes and support for
user-defined macros, but for now it would be possible to work only with a set
of hard-coded annotation macros.
implementation of this requires both parser changes and support for user-defined
macros, but for now it would be possible to work only with a set of hard-coded
annotation macros.
Annotations can be arbitrarily nested, so a set of annotation macros become
implicitly nested inside each other:
@ -60,11 +62,12 @@ derive Eq Debug
```
In the presence of annotations and macros, it becomes more and more important
that we are able to reserve words such as `type` to ensure that users can
always have a good sense of what the most common constructs in the language
mean, rather than allowing them to be overridden outside of the stdlib.
that we are able to reserve words such as `type` to ensure that users can always
have a good sense of what the most common constructs in the language mean,
rather than allowing them to be overridden outside of the stdlib.
## Automatic Deriving
In order to make the language easier to debug, we have all types automatically
derive an interface `DebugShow`. This interface provides a function that will
print all the significant information about the value (e.g. locations, types,

24
docs/syntax/naming.md
View File

@ -7,6 +7,7 @@ order: 2
---
# Naming
This file describes the syntax for naming language constructs in Enso, as well
as the various rules that names follow.
@ -26,6 +27,7 @@ giving Enso code a uniform identity.
<!-- /MarkdownTOC -->
## Naming Constructs
Given that Enso is dependently-typed, with no artificial separation between the
type and value-level syntaxes, an arbitrary name can refer to both types and
values. This means that naming itself can become a bit of a concern. At first
@ -37,9 +39,9 @@ As we still want to have a minimal syntax for such use-cases, Enso enforces the
following rules around naming:
- All identifiers are named as follows. This is known as 'variable' form.
+ Each 'word' in the identifier must be lower-case or a number.
+ Words in the identifier are separated using `_`.
+ Numbers may not occur as the first 'word' in an identifier.
- Each 'word' in the identifier must be lower-case or a number.
- Words in the identifier are separated using `_`.
- Numbers may not occur as the first 'word' in an identifier.
- An identifier named as above can be referred to by capitalizing the first
letter of each 'word' in the identifier. This is known as 'referent' form.
- No mixed-format names are allowed (e.g. `HTTP`, `foO`, `make_New`, or
@ -53,10 +55,10 @@ Name resolution obeys the following rules:
- In a [pattern context](#pattern-contexts), an identifier in referent form will
_always_ refer to a name in scope, whereas an identifier in variable form is
interpreted as the creation of a fresh identifier.
- This behaviour _only_ occurs in pattern contexts. In all other contexts,
both conventions refer to that name already in scope.
- Operator names behave as variable names when placed in a prefix position
(e.g. `+ a b`).
- This behaviour _only_ occurs in pattern contexts. In all other contexts, both
conventions refer to that name already in scope.
- Operator names behave as variable names when placed in a prefix position (e.g.
`+ a b`).
- Operator names behave as referent names when placed in an infix position (e.g.
`a + b`).
- All literals (e.g. `1` and `"Hello"`) are treated as referent names.
@ -68,6 +70,7 @@ Identifiers are introduced by:
- Using them in a type ascription (free variables).
## Pattern Contexts
A pattern context is a span in the code where variable identifiers (as described
above) can be used to introduce new identifiers into the scope containing the
pattern context. The following spans are pattern contexts:
@ -99,6 +102,7 @@ means that these are the _only_ constructs which introduce pattern contexts.
> means in a formal sense.
## Localised Naming
We do, however, recognise that there is sometimes a need for unicode characters
in names (e.g. designing a high-level visual library that targets a narrow
domain in a specific country). To that end, Enso allows users to specify
@ -108,6 +112,7 @@ Special support is provided for providing completions based on these localised
names in the language server, and in Enso Studio.
## Operator Naming
Operator names are those built solely from operator symbols (e.g. `+` or `<*>`).
Operator symbols are defined as characters in the following set.
@ -120,6 +125,7 @@ _valid_ operator name, as some may collide with built-in language constructs
(e.g. `[` and `]`, which start and end a vector literal respectively).
## Reserved Names
Even though we do not intend to reserve any names at the level of the lexer or
parser, there are a number of constructs so core to the operation of Enso as a
language that we do not want to let them be overridden or redefined by users.
@ -140,8 +146,8 @@ the readability and consistency of Enso code. They are as follows:
- `.`: This is the forward function chaining operator.
- `case ... of`: This reserved name is the case expression that is fundamental
to the operation of control flow in the language.
- `this`: This reserved name is the one used to refer to the enclosing type in
a method or type definition.
- `this`: This reserved name is the one used to refer to the enclosing type in a
method or type definition.
- `here`: This reserved name is the one used to refer to the enclosing module.
- `in`: Used to specify the monadic context(s) in a type signature.

8
docs/syntax/projections.md
View File

@ -7,6 +7,7 @@ order: 12
---
# Projections and Field Access
Enso provides multiple ways for users to access data from their types. It has
the old functional stalwart of pattern matching, but it also has an inbuilt
notion of accessors based on lenses (field projections).
@ -20,6 +21,7 @@ notion of accessors based on lenses (field projections).
<!-- /MarkdownTOC -->
## Pattern Matching
Pattern matching in Enso works similarly to as you would expect in various other
functional languages. Typing information is _always_ refined in the branches of
a case expression, which interacts well with dependent typing and type-term
@ -68,16 +70,18 @@ unification. There are a few main ways you can pattern match:
> The actionables for this section :
>
> - Refine the syntax for the name-based case.
> - Provide code examples for why the renaming use-case is important (e.g.
> cases where there are clashing field names).
> - Provide code examples for why the renaming use-case is important (e.g. cases
> where there are clashing field names).
> - Function-resolution matching.
### The Underscore in Pattern Matching
An underscore `_` passed as an argument to a syntactic pattern does not behave
like the function argument shorthand. Instead it acts as a positional match that
is given no name.
## Projections
Unlike the simple accessors defined by most programming language, Enso's
accessors are far more powerful. This is because they are based on lenses.

39
docs/syntax/top-level.md
View File

@ -7,6 +7,7 @@ order: 9
---
# Enso Top-Level Syntax
Like almost all statically-typed programming languages, the top-level of an Enso
file is non-executable. The top level may contain the following constructs:
@ -26,6 +27,7 @@ file is non-executable. The top level may contain the following constructs:
<!-- /MarkdownTOC -->
## Main
The entry point for an Enso program is defined in a special top-level binding
called `main` in the file `Main.enso`. However, we also provide for a scripting
workflow in the form of `enso run`, which will look for a definition of `main`
@ -36,33 +38,34 @@ main = IO.println "Hello, World!"
```
## Against Top-Level Evaluation
At points during Enso's development it was up for debate as to whether we wanted
the language to have an executable top-level (akin to scripting languages like
Python). In order to make a decision, we listed the following use cases, and the
corresponding analysis is provided here for posterity.
| Label | Meaning |
| --------| ------- |
| `[?,_]` | We don't know how to implement it, but it may be possible.
| `[-,_]` | Not possible to implement using purely syntactic macros.
| `[M,_]` | Possible to implement using purely syntactic macros.
| `[_,H]` | High priority. This will be used often.
| `[_,M]` | Medium priority. This will be used with a medium frequency.
| `[_,L]` | Low priority. Nice to have, but we can likely live without it.
| `[_,!]` | Something that we never want to have in the language.
| Label | Meaning |
| ------- | -------------------------------------------------------------- |
| `[?,_]` | We don't know how to implement it, but it may be possible. |
| `[-,_]` | Not possible to implement using purely syntactic macros. |
| `[M,_]` | Possible to implement using purely syntactic macros. |
| `[_,H]` | High priority. This will be used often. |
| `[_,M]` | Medium priority. This will be used with a medium frequency. |
| `[_,L]` | Low priority. Nice to have, but we can likely live without it. |
| `[_,!]` | Something that we never want to have in the language. |
The use-cases we have considered are as follows:
| Label | Description |
| ------- | ----------- |
| `[-,L]` | Creating top-level constructs in `IO`, such as `IORef`. This is, in general, considered to be bad style, but can sometimes be useful. |
| `[-,L]` | Using enso files like python is able to be for scripting work. The ability to write constructs at the top-level and just evaluate them. |
| `[M,H]` | The ability to generate structures and / types for a dataframe at compilation time, or the automatic generation of an API for a library. A key recognition is that dependent types and type-level execution replace much of the need to be able to query the type-checker and runtime while writing a syntactic macro. |
| `[M,H]` | Static metaprogramming (transformations from `AST -> AST`) to let users generate types and functions based on existing AST. There is the potential to want to be able to evaluate actions in `IO` while doing this, but it may not be necessary. |
| `[-,!]` | Dynamic metaprogramming to let users mutate program state at runtime (e.g. changing atom shapes, function definitions), also known as 'monkey patching'. This is not something we want in the language, but we do perhaps want the ability to do so on values of type `Dynamic`. |
| Label | Description |
| ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `[-,L]` | Creating top-level constructs in `IO`, such as `IORef`. This is, in general, considered to be bad style, but can sometimes be useful. |
| `[-,L]` | Using enso files like python is able to be for scripting work. The ability to write constructs at the top-level and just evaluate them. |
| `[M,H]` | The ability to generate structures and / types for a dataframe at compilation time, or the automatic generation of an API for a library. A key recognition is that dependent types and type-level execution replace much of the need to be able to query the type-checker and runtime while writing a syntactic macro. |
| `[M,H]` | Static metaprogramming (transformations from `AST -> AST`) to let users generate types and functions based on existing AST. There is the potential to want to be able to evaluate actions in `IO` while doing this, but it may not be necessary. |
| `[-,!]` | Dynamic metaprogramming to let users mutate program state at runtime (e.g. changing atom shapes, function definitions), also known as 'monkey patching'. This is not something we want in the language, but we do perhaps want the ability to do so on values of type `Dynamic`. |
| `[M,H]` | 'Remembering' things when compiling a file, such as remembering all structures marked by an `AST` annotation. An example use case for a mechanism like this is to generate pattern matches for all possible `AST` types. This can be done by letting macros write to a per-file peristent block of storage that could be serialised during precompilation. |
| `[M,H]` | Grouping of macros (e.g. `deriveAll = derive Ord Debug Show`). This can be easily handled by doing discovery on functions used as macros, and treating it as a macro as well. |
| `[?,M]` | Method-missing magic, akin to ruby. This is likely able to be handled using other, existing language mechanisms. |
| `[M,H]` | Grouping of macros (e.g. `deriveAll = derive Ord Debug Show`). This can be easily handled by doing discovery on functions used as macros, and treating it as a macro as well. |
| `[?,M]` | Method-missing magic, akin to ruby. This is likely able to be handled using other, existing language mechanisms. |
In summary and when considering the above use-cases, it seems that there is
little need for top-level expression evaluation in Enso. We can support all of

56
docs/syntax/types.md
View File

@ -7,6 +7,7 @@ order: 7
---
# Types and Type Signatures
Enso is a statically typed language, meaning that every variable is associated
with information about the possible values it can take. In Enso, the type
language is the same as the term language, with no artificial separation. For
@ -37,9 +38,10 @@ is a useful way of thinking about things when discussing type signatures.
<!-- /MarkdownTOC -->
## Type Signatures
Enso allows users to provide explicit type signatures for values through use of
the type ascription operator `:`. The expression `a : b` says that the value
`a` has the type `b` attributed to it.
the type ascription operator `:`. The expression `a : b` says that the value `a`
has the type `b` attributed to it.
```ruby
foo : (m : Monoid) -> m.this
@ -64,24 +66,25 @@ Type signatures in Enso have some special syntax:
```
### Type Operators
Please note that `:`, `in`, and `!` all behave as _standard operators_ in Enso.
This means that you can section them, which is incredibly useful for programming
with types. In addition, Enso supports a number of additional operators for
working with types. These are listed below.
| Operator | Precedence Relations | Level | Assoc. | Description |
|:--------:|:-----------------------------:|:-----:|:------:|:----------------------------------------------------------------------------|
| `:` | `> =` | 0 | Left | Ascribes the type (the right operand) to the value of the left operand. |
| `in` | `> :`, `> !` | 3 | Left | Ascribes the context (the right operand) to the value of the left operand. |
| `!` | `> :`, `> ->` | 2 | Left | Combines the left operand with the right operand as an error value. |
| `->` | `> :` | 1 | Left | Represents a mapping from the left operand to the right operand (function). |
| `<:` | `> !`, `< \|`, `> in` | 4 | Left | Asserts that the left operand is structurally subsumed by the right. |
| `~` | `== <:` | 4 | Left | Asserts that the left and right operands are structurally equal. |
| `;` | `< :`, `> =` | -2 | Left | Concatenates the left and right operand typesets to create a new typeset. |
| `\|` | `> <:`, `> !`, `> in`, `> :` | 5 | Left | Computes the union of the left and right operand typesets. |
| `&` | `> \|` | 6 | Left | Computes the intersection of the left and right operand typesets. |
| `\` | `> &` | 7 | Left | Computes the subtraction of the right typeset from the left typeset. |
| `:=` | `< :`, `> =`, `> ;` | -1 | Left | Creates a typeset member by assigning a value to a label. |
| Operator | Precedence Relations | Level | Assoc. | Description |
| :-------------------------------------------------------------------------------------------: | :--------------------------: | :---: | :----: | :-------------------------------------------------------------------------- |
| `:` | `> =` | 0 | Left | Ascribes the type (the right operand) to the value of the left operand. |
| `in` | `> :`, `> !` | 3 | Left | Ascribes the context (the right operand) to the value of the left operand. |
| `!` | `> :`, `> ->` | 2 | Left | Combines the left operand with the right operand as an error value. |
| `->` | `> :` | 1 | Left | Represents a mapping from the left operand to the right operand (function). |
| `<:` | `> !`, `< \|`, `> in` | 4 | Left | Asserts that the left operand is structurally subsumed by the right. |
| `~` | `== <:` | 4 | Left | Asserts that the left and right operands are structurally equal. |
| `;` | `< :`, `> =` | -2 | Left | Concatenates the left and right operand typesets to create a new typeset. |
| `\|` | `> <:`, `> !`, `> in`, `> :` | 5 | Left | Computes the union of the left and right operand typesets. |
| `&` | `> \|` | 6 | Left | Computes the intersection of the left and right operand typesets. |
| `\` | `> &` | 7 | Left | Computes the subtraction of the right typeset from the left typeset. |
| `:=` | `< :`, `> =`, `> ;` | -1 | Left | Creates a typeset member by assigning a value to a label. |
Solving this set of inequalities produces the _relative_ precedence levels for
these operators shown in the table above. In order to check this, you can use
@ -137,13 +140,15 @@ A permalink to the program using an online Z3 console can be found
> - Decide which of these should be exposed in the surface syntax.
### Typeset Literals
Sometimes it is useful or necessary to write a typeset _literal_ in your code.
These work as follows.
- **Typeset Member:** Syntax for typeset members have three components:
+ **Label:** The name of the member. This must always be present.
+ **Type:** The type of the member. This need not be present.
+ **Value:** A value for the member. This need not be present.
- **Label:** The name of the member. This must always be present.
- **Type:** The type of the member. This need not be present.
- **Value:** A value for the member. This need not be present.
This looks like the following:
@ -175,6 +180,7 @@ Typeset literals are considered to be a
apply.
### Writing Type Signatures
When ascribing a type to a value, there are two main ways in which it can be
done. Both of these ways are _semantically_ equivalent, and ascribe the type
given by the signature (to the right of the `:`) to the expression to the left
@ -187,9 +193,9 @@ of the `:`.
my_expr : Type
```
2. **Freestanding Ascription:** Using the type ascription operator to
associate a type with a name. The name must be defined on _the line below_
the ascription.
2. **Freestanding Ascription:** Using the type ascription operator to associate
a type with a name. The name must be defined on _the line below_ the
ascription.
```ruby
a : Type
@ -211,6 +217,7 @@ of the `:`.
> directly adjacent to the ascribed value?
### Behaviour of Type Signatures
In Enso, a type signature operates to constrain the values that a given variable
can hold. Type signatures are _always_ checked, but Enso may maintain more
specific information in the type inference and checking engines about the type
@ -244,6 +251,7 @@ properties:
> - Does this differ for root and non-root definitions?
## Operations on Types
Enso also provides a set of rich operations on its underlying type-system notion
of typesets. Their syntax is as follows:
@ -260,10 +268,11 @@ of typesets. Their syntax is as follows:
> supersedes this section while this actionable exists.
## Type Definitions
Types in Enso are defined by using the `type` reserved name. This works in a
context-dependent manner that is discussed properly in the
[type system design document](../types/README.md), but is summarised
briefly below.
[type system design document](../types/README.md), but is summarised briefly
below.
- **Name and Fields:** When you provide the keyword with only a name and some
field names, this creates an atom.
@ -314,6 +323,7 @@ type Monoid
```
### Visibility and Access Modifiers
While we don't usually like making things private in a programming language, it
sometimes the case that it is necessary to indicate that certain fields should
not be touched (as this might break invariants and such like). To this end, we

17
docs/types/README.md
View File

@ -7,6 +7,7 @@ order: 0
---
# Enso's Type System
On the spectrum of programming type systems ranging from dynamically typed to
statically typed, one likes to think that there is a happy medium between the
two. A language that _feels_ dynamic, with high levels of type inference, but
@ -31,19 +32,19 @@ many syntactic language features upon inference and type checking, and is
instrumental for ensuring that we build the right language.
> #### A Note on Syntax
>
> In the aid of precision, this document will use syntax that _may not_ be
> exposed to users. The appearance of a piece of syntax here that is not
> described in the [syntax](../syntax/README.md) document makes no promises as
> described in the [syntax](../syntax/README.md) document makes no promises as
> to whether said syntax will be exposed in the surface language.
> **Please Note:**
> The designs in this section are currently very exploratory as the type system
> is not slated from implementation until after 2.0.
> **Please Note:** The designs in this section are currently very exploratory as
> the type system is not slated from implementation until after 2.0.
Information on the type system is broken up into the following sections:
- [**Goals for the Type System:**](./goals.md) The goals for the
Enso type system, particularly around usability and user experience.
- [**Goals for the Type System:**](./goals.md) The goals for the Enso type
system, particularly around usability and user experience.
- [**The Type Hierarchy:**](./hierarchy.md) The type hierarchy in Enso.
- [**Function Types:**](./function-types.md) Function types in Enso.
- [**Access Modification:**](./access-modifiers.md) Access modifiers in Enso
@ -60,8 +61,8 @@ Information on the type system is broken up into the following sections:
- [**Analysing Parallelism:**](./parallelism.md) A description of how the type
system interacts with the planned automated parallelism analysis.
- [**Type-Directed Programming:**](./type-directed-programming.md) A description
of how the type system aids type-directed programming, and the features it
has to support this approach.
of how the type system aids type-directed programming, and the features it has
to support this approach.
- [**Errors:**](./errors.md) The interaction between Enso's errors and the type
system.
- [**Type Inference and Checking:**](./inference-and-checking.md) A description

4
docs/types/access-modifiers.md
View File

@ -7,6 +7,7 @@ order: 4
---
# Access Modifiers
While we don't usually like making things private in a programming language, it
sometimes the case that it is necessary to indicate that certain fields should
not be touched (as this might break invariants and such like). To this end, Enso
@ -21,6 +22,7 @@ provides an explicit mechanism for access modification.
<!-- /MarkdownTOC -->
## Access Modification
Access modifiers in Enso work as follows:
- We have a set of access modifiers, namely `private` and `unsafe`.
@ -66,6 +68,7 @@ Access modifiers in Enso work as follows:
> - How do we type this?
## Private
The `private` modifier acts to hide implementation details from clients of the
API. It is:
@ -73,6 +76,7 @@ API. It is:
- Able to be avoided using the above-described mechanism.
## Unsafe
While `private` works as you might expect, coming from other languages, the
`unsafe` annotation has additional restrictions:

12
docs/types/contexts.md
View File

@ -7,6 +7,7 @@ order: 8
---
# Monadic Contexts
Coming from a Haskell background, we have found that Monads provide a great
abstraction with which to reason about program behaviour, but they have some
severe usability issues. The main one of these is the lack of automatic lifting,
@ -35,12 +36,13 @@ the compiler, and hence can be automatically lifted to aid usability.
- [Context Definitions](#context-definitions)
- [Context Lifting](#context-lifting)
- [Inbuilt Contexts](#inbuilt-contexts)
- [IO](#io)
- [State](#state)
- [IO](#io)
- [State](#state)
<!-- /MarkdownTOC -->
## Context Syntax
There are three main notes about the syntax of contexts:
1. Monadic contexts are defined using the `in` keyword (e.g. `Int in IO`).
@ -50,6 +52,7 @@ There are three main notes about the syntax of contexts:
through repeated uses of `in`.
## Monadic Bind
It is also important to note that Enso has no equivalent to `<-` in Haskell.
Instead, pure computations are implicitly placed in the `Pure` monadic context,
and `=` acts to 'peel off' the outermost layer of contexts. As such, this means
@ -57,6 +60,7 @@ that `=` _always_ acts as `bind`, greatly simplifying how the type-checker has
to work.
## Context Definitions
Contexts can be defined by users.
> The actionables for this section are:
@ -64,12 +68,14 @@ Contexts can be defined by users.
> - How, what, when and why?
## Context Lifting
> The actionables for this section are:
>
> - Specify and explain how automated lifting of monadic contexts works.
> - It depends on the order of `runCtx`
## Inbuilt Contexts
Enso includes a set of commonly-used monadic contexts as part of `Base`, its
standard library. These are listed below.
@ -78,6 +84,7 @@ standard library. These are listed below.
> - Determine the full set of contexts that Enso should provide by default.
### IO
> The actionables for this section are:
>
> - Determine the granularity of IO (it's not one context, but a lot).
@ -85,6 +92,7 @@ standard library. These are listed below.
> as well as the impacts of this.
### State
> The actionables for this section are:
>
> - Determine exactly how state works (the fact that the 'keys' are preset by

32
docs/types/dependent-typing.md
View File

@ -7,6 +7,7 @@ order: 14
---
# Dependent Typing
Enso is a [dependently typed](https://en.wikipedia.org/wiki/Dependent_type)
programming language. This means that types are first-class values in the
language, and hence can be manipulated and computed upon just like any other
@ -37,6 +38,7 @@ to automate much of the proof burden through SMT solvers.
<!-- /MarkdownTOC -->
## Proving Program Properties
Some notes:
- Dependent types as constructing proofs through combining types. Combining
@ -53,25 +55,26 @@ Some notes:
and utilises this when typechecking. It may also take a string description of
the property to prove, allowing for nicer error messages:
```
append : (v1 : Vector a) -> (v2 : Vector a) -> (v3 : Vector a)
append = vec1 -> vec2 ->
prove (v3.size == v1.size + v2.size) "appending augments length"
...
````
```
append : (v1 : Vector a) -> (v2 : Vector a) -> (v3 : Vector a)
append = vec1 -> vec2 ->
prove (v3.size == v1.size + v2.size) "appending augments length"
...
```
Sample error:
Sample error:
```
[line, col] Unable to prove that "appending augments length":
Required Property: v3.size == v1.size + v2.size
Proof State: <state>
```
[line, col] Unable to prove that "appending augments length":
Required Property: v3.size == v1.size + v2.size
Proof State: <state>
<caret diagnostics>
```
<caret diagnostics>
```
This gives rise to the question as to how we determine which properties (or
data) are able to be reasoned about statically.
- Dependent types in Enso will desugar to an application of Quantitative Type
Theory.
@ -80,9 +83,10 @@ Some notes:
> - Specify how we want dependency to behave in a _far more rigorous_ fashion.
## Automating the Proof Burden
Even with as capable and simple a dependently-typed system as that provided by
Enso, there is still a burden of proof imposed on our users that want to use
these features. However, the language [F*](https://www.fstar-lang.org/) has
these features. However, the language [F\*](https://www.fstar-lang.org/) has
pioneered the combination of a dependently-typed system with an SMT solver to
allow the automation of many of the simpler proofs.

Some files were not shown because too many files have changed in this diff Show More