enso/app/gui/docs/CONTRIBUTING.md

layout	title	category	tags
developer-doc	Development & Contributing Guide	summary	summary contributing

Development & Contributing Guide

Thank you for your interest in contributing to the Enso IDE! We believe that only through community involvement can Enso be the best it can be! There are a whole host of ways to contribute, and every single one is appreciated.

Reporting Issues

If you are concerned that your bug publicly presents a security risk to the users of Enso, please contact security@enso.org.

While it's never great to find a bug, they are a reality of software and software development! We can't fix or improve on the things that we don't know about, so report as many bugs as you can! If you're not sure whether something is a bug, file it anyway!

Even though GitHub search can sometimes be a bit hard to use, we'd appreciate if you could search for your issue before filing a bug as it's possible that someone else has already reported the issue. We know the search isn't the best, and it can be hard to know what to search for, so we really don't mind if you do submit a duplicate!

Opening an issue is as easy as following this link and filling out the fields. The template is intended to collect all the information we need to best diagnose the issue, so please take the time to fill it out accurately.

The reproduction steps are particularly important, as the more easily we can reproduce it, the faster we can fix the bug! It's also helpful to have the version of the IDE, as that will let us know if the bug is Operating System or Architecture specific.

Development Environment

The project builds on macOS, Windows, and Linux. Cross-platform targets work well on all of these platforms, however, macOS package will miss the right application icon if built on Linux or Windows due to non-trivial icon generation on these platforms. To develop the source code you will need the following setup:

• The Rust Toolchain (nightly-2021-11-29)

  This project uses several features available only in the nightly Rust toolchain. Please use the the Rust toolchain installer to manage Rust toolchains. It will automatically download the toolchain needed to build the project.

  In addition, some custom CLI programs need to be installed manually:

  rustup toolchain install stable                  # Stable toolchain required for the following tools.
  cargo +stable install wasm-pack --version 0.10.2 # Install the wasm-pack toolkit.
  cargo +stable install cargo-watch                # To enable ./run watch utility
  

  Make sure that your PATH environment variable is set up correctly, so that the binaries installed by cargo (typically to $HOME/.cargo/bin) can be run from the command line; verify this by running wasm-pack --version.

• Node and Node Package Manager LTS

  To build the web and desktop applications you will need the latest LTS version of node and npm. Even minor release changes are known to cause serious issues, thus we provide support for the latest LTS version only. Please do not report build issues if you use other versions. In case you run macOS or Linux the easiest way to set up the proper version is by installing the Node Version Manager and running nvm install --lts && nvm use --lts.

• (Optional) FlatBuffer compiler flatc

  This dependency is needed only if you need to update files generated by the FlatBuffer from the Engine Services binary protocol description. Otherwise, relying on the generated files that are being stored in this repository is fine.

  flatc must be in the version newer than 1.12 due to this bug. As of writing this text there are no official releases with this issue fixed, however current binaries can be obtained from the project's CI build artifacts. flatc builds from 8 May 2020 onwards have been confirmed to work.

  After placing flatc in PATH you need to define the ENSO_IDE_ENABLE_FLATC environment variable to explicitly enable regeneration of the interface files. The flatc is run as part of build.rs script of the `enso-protocol package.

Working with sources

Please be sure to carefully read the Rust style guide 1 and the Rust style guide 2 before contributing to the codebase.

We do also use prettier for the JavaScript and Markdown files in our code base. If you have not installed it already you can do so via npm install prettier. To use it manually via command line run prettier --write to all JavaScript files in the project. Alternatively, there are plugins for many IDEs available to do this for you.

Repository Structure Overview

Note: Currently, the Enso repository is going through a process of refactoring, and it is not finished yet: the Engine files are still not in the app/engine where they ought to be, but in the root directory instead.

The root directory contains Cargo.toml and build.sbt files, allowing to open all rust or scala code as a single project in your favorite IDE. There is also a run script used for building and running the Enso IDE (see next section for details).

The subdirectories of interests are:

• app: The actual products delivered in this repository:
  • gui: A rust crate compiled to a WASM library with all the logic of the GUI layer. The library is used by both the desktop application and the cloud environment. For further documentation see the documentation of the crate (at the top of the src/lib.rs file).
  • ide-desktop: The desktop version of the Enso IDE. Implemented as an electron application which spawns backend services, loads the WASM gui library and runs the main entry point.
  • engine: (In the future: see the note at the section beginning). The implementation of the language itself: CLI tools like compiler or interpreter, as well as the services used as a backend for the Enso IDE (Language Server and Project Manager).
• lib: All libraries not being the main components of our application. They are grouped by language. The most prominent are:
  • rust/prelude: A library containing the most popular utilities and imports. Should be imported in each rust module - see Contributing guidelines.
  • rust/ensogl: EnsoGL Framework for creating efficient GUI applications in WASM.
  • rust/frp: The library allows following the Functional Reactive Programming paradigm in rust.
• build: All building scripts and utilities, mostly the logic of the ./run script.

Other directories are auto-generated dist and target, or (currently) are the Engine files, which will be moved to app/engine soon.

Development

As this is a multi-part project with many complex dependencies, it is equipped with a build script which both validates your working environment and takes care of providing the most suitable compilation flags for a particular development stage. To run the build script simply run node ./run in the root of the codebase. On macOS and Linux you can use a simpler form of ./run, however, this doc will use the former form in order to stay cross-platform compatible. Run node ./run help to learn about available commands and options. All arguments provided after the -- symbol will be passed to sub-commands. For example node ./run build -- --dev will pass the --dev flag to cargo (Rust build tool). The most common options are presented below:

• Interactive mode Run node ./run watch to start a local web-server and a source-file watch utility which will build the project on every change. Open http://localhost:8080 (the port may vary and will be reported in the terminal if 8080 was already in use) to run the application, or http://localhost:8080/?entry to open example demo scenes list. Please remember to disable the cache in your browser during the development! By default, the script disables heavyweight optimizations to provide interactive development experience. The scripts are thin wrappers for wasm-pack and accept the same command line arguments.

• Production mode In order to compile in a production mode (enable all optimizations, strip WASM debug symbols, minimize the output binaries, etc.), run node ./run build. To create platform-specific packages and installers use node ./run dist instead. The final executables will be located at dist/client/$PLATFORM.

• Selective mode In order to compile only part of the project, and thus drastically shorten the incremental compile time, you are advised to use the selective compilation mode by passing the --crate option to the build or watch command, e.g. node ./watch --crate ensogl/example to compile only the renderer-related example scenes. Please note, that in order to run a scene in a web-browser, the scene has to be compiled and has to expose a public function with a name starting with entry_point_. Thus, if you compile only selected crate, you will have access only to the example scenes that were defined or re-exported by that crate. In particular, the ide crate exposes the entry_point_ide function, so you have to compile it to test your code in the Enso IDE.

Updating JS dependencies

JS parts of the project are using lerna to manage JS dependencies. When you add a new JS dependency or update an old one to the new version, lerna must be re-run to trigger npm install for all projects. The easiest way to do so is to run ./run clean --no-rust command. You can also manually call npm run install in app/ide-desktop/ directory with a similar result. Our CI does that for every build at the moment.

Testing IDE with a specific version of a backend

Sometimes changes to the IDE must be tested against the unreleased Enso Engine version. To perform this one should:

• commit code to enso repository and allow CI to successfully complete;
• enter the Engine CI runs and select the run for the commit to be tested;
• download the Enso Engine artifact appropriate for your OS (e.g. enso-engine-0.2.9-SNAPSHOT-windows-amd64.zip);
• extract the archive into ENSO_DATA_DIRECTORY/dist. Rename the extracted folder by erasing initial enso-, e.g. change enso-0.2.9-SNAPSHOT to 0.2.9-SNAPSHOT;
• open the package.yaml of the tested project (by default it is under ~/enso/projects/Unnamed/) and change the enso-version value to the version of the downloaded package (e.g. enso-version: 0.2.9-SNAPSHOT);
• run the IDE opening the updated project. Project Manager should automatically spawn the Language Server from the extracted package.

Using IDE as a library.

In case you want to use the IDE as a library, for example to embed it into another website, you need to first build it using node ./run {built,dist} and find the necessary artifacts located at dist/content. Especially, the dist/content/index.js defines a function window.enso.main(cfg) which you can use to run the IDE. Currently, the configuration argument can contain the following options:

• entry - the entry point, one of predefined scenes. Set it to empty string to see the list of possible entry points.
• project - the project name to open after loading the IDE.

Testing, Linting, and Validation

After changing the code it's always a good idea to lint and test the code. We have prepared several scripts which maximally automate the process:

• Size Validation Use node ./run build to check if the size of the final binary did not grew too much in comparison to the previous release. Watching the resulting binary size is one of the most important responsibility of each contributor in order to keep the project small and suitable for web-based usage. In case the size will exceed the limits:

  • If the PR does not include any new libraries, you are allowed to increase the limit by 10KB. In case the limit will be exceeded by more than 10KB, check which part of the code contributet to it, and talk about it with the code owner.
  • If the PR does include new libraries, you are allowed to increase the limit by 10KB, but you should also consider if it is possible to get the same results without using a new library (even by implementing few lines of code from the library in sources of the project).
  • If the PR does include new libraries, and the limit is exceeded by more than 10KB, check which part of the code contributed to it, and talk about it with the code owner.
  • If the PR does include new libraries, and the limit is exceeded by more than 50KB, it would probably not be merged. Research possible alternatives before talking with code owner about this case.

• Testing For the test suite to run you need a current version of Chrome installed. Use node ./run test run both unit and web-based visual test.

  • Note for Windows users: there is a known issue with wasm-pack using the wrong version of the chrome driver. There is a workaround described in the issue: download compatible ChromeDriver from the official source and ensure it is in your PATH.

• Linting Please be sure to fix all errors reported by node ./run lint before creating a pull request to this repository.

Development Branches

The following branches are used to develop the product:

• wip/[initials]/[feature]
  Feature branches. These are temporary branches used by the team to develop a particular feature.

• develop
  Contains the most recent changes to the product. After successful review, the feature branches are merged here. Each commit to this branch will result in a nightly build of the product accessible as CI artifacts.

• unstable
  Contains only those commits which can be considered unstable product releases. Each commit to this branch will result in an unstable release of the product and will be published on GitHub as a pre-release. The build version and build description will be automatically fetched from the newest CHANGELOG.md entry and will fail if the version will not be of the form [major].[minor].[patch]-[sfx], where [sfx] is one of alpha.[n], beta.[n], or rc.[n], where [n] is an unstable build number.

• stable Contains only those commits which can be considered stable product releases. Each commit to this branch will result in a stable release of the product and will be published on GitHub as a release. The build version and build description will be automatically fetched from the newest CHANGELOG.md entry and will fail if the version will not be of the form [major].[minor].[patch].

Forcing CI builds

By default, CI would not build artifacts from wip and develop branches in order to save time and resources. If you want the artifacts to be build for your PR, simply add [ci build] anywhere in the PR description.

Skipping CHANGELOG.md change assertions

By default, CI would fail if the CHANGELOG.md file does not need to be updated. However, sometimes there are PRs that does not change anything significant in the final product. You can then simply add [ci no changelog needed] anywhere in your PR description to skip this assertion.

Publishing Results

All new changes should be proposed in the form of Pull Requests (PRs) to this repository. Each PR should contain changes to documentation and CHANGELOG.md if applicable.

Changelog

Please remember to update the CHANGELOG.md on every new bug fix or feature implementation. Please note that CHANGELOG.md is used to establish the current product version (the run script extracts it from the newest changelog entry). Thus, be sure to always increase the newest version in the changelog after a release, otherwise CI will fail. Please use the docs/CHANGELOG_TEMPLATE.md as the template to create new changelog entries. Please note, that there is a special syntax for defining features of the upcoming release. The newest changelog entry can have a title "Next Release". In such a case, the build version will be 0.0.0 and CI would fail when trying to publish it as a release.