roc/crates/README.md

# Roc Internals

Roc has different rust crates for various binaries and libraries. These are described below to help give you an overview. If you see something here that is out of date, please correct it or discuss it in the [Zulip chat](https://roc.zulipchat.com/).

Use `cargo doc` to generate docs; e.g. ```cargo doc --package roc_ast --open```.

## `ast/` - `roc_ast`

Code to represent the [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) as used by the editor.
In contrast to the compiler, the types in this AST do not keep track of the location of the matching code in the source file.


## `cli/` - `roc_cli`

The `roc` binary that brings together all functionality in the Roc toolset.

## `cli_utils/` - `cli_utils`

Provides shared code for cli tests and benchmarks.

## `code_markup/` - `roc_code_markup`

Roc's very own [markup language](https://en.wikipedia.org/wiki/Markup_language). This library is used by the editor to display good-looking roc code. 

## `compile/`

Compiles `.roc` files and combines them with their platform into an executable binary. See [compiler/README.md](./compiler/README.md) for more information.

TODO explain what "compiler frontend" is
TODO explain what "compiler backend" is

The compiler includes the following sub-crates;
- `roc_alias_analysis` Performs analysis and optimizations to remove unneeded reference counts at runtime, and supports in-place mutation.
- `arena-pool` An implementation of an arena allocator designed for the compiler's workloads.
- `roc_build` Responsible for coordinating building and linking of a Roc app with its host.
- `roc_builtins` provides the Roc functions and modules that are implicitly imported into every module. See [README.md](./compiler/builtins/README.md) for more information.
- `roc_can` Canonicalize a roc abstract syntax tree, resolving symbols, re-ordering definitions, and preparing a module for type inference.
- `roc_collections` Domain-specific collections created for the needs of the compiler.
- `roc_constrain` Responsible for building the set of constraints that are used during type inference of a program, and for gathering context needed for pleasant error messages when a type error occurs.
- `roc_debug_flags` Environment variables that can be toggled to aid debugging of the compiler itself.
- `roc_derive` provides auto-derivers for builtin abilities like `Hash` and `Decode`.
- `roc_exhaustive` provides exhaustiveness checking for Roc 
- `roc_fmt` The roc code formatter.
- `roc_gen_dev` provides the compiler backend to generate Roc binaries fast, for a nice developer experience. See [README.md](./compiler/gen_dev/README.md) for more information.
- `roc_gen_llvm` provides the LLVM backend to generate Roc binaries. Used to generate a binary with the fastest possible execution speed.
- `roc_gen_wasm` provides the WASM backend to generate Roc binaries. See [README.md](./compiler/gen_wasm/README.md) for more information.
- `roc_ident` Implements data structures used for efficiently representing small strings, like identifiers.
- `roc_intern` provides generic interners for concurrent and single-thread use cases.
- `roc_late_solve` provides type unification and solving primitives from the perspective of the compiler backend.
- `roc_load` Used to load a .roc file and coordinate the compiler pipeline, including parsing, type checking, and code generation.
- `roc_load_internal` The internal implementation of roc_load, separate from roc_load to support caching.
- `roc_module` Implements data structures used for efficiently representing unique modules and identifiers in Roc programs.
- `roc_mono` Roc's main intermediate representation (IR), which is responsible for monomorphization, defunctionalization, inserting ref-count instructions, and transforming a Roc program into a form that is easy to consume by a backend.
- `roc_parse` Implements the Roc parser, which transforms a textual representation of a Roc program to an abstract syntax tree.
- `roc_problem` provides types to describe problems that can occur when compiling `.roc` code.
- `roc_region` Data structures for storing source-code-location information, used heavily for contextual error messages.
- `roc_target` provides types and helpers for compiler targets such as `default_x86_64`.
- `roc_serialize` provides helpers for serializing and deserializing to/from bytes.
- `roc_solve` The entry point of Roc's type inference system. Implements type inference and specialization of abilities.
- `roc_solve_problem` provides types to describe problems that can occur during solving.
- `roc_str` provides `Roc` styled collection reference counting. See [README.md](./compiler/str/README.md) for more information.
- `test_derive` Tests Roc's auto-derivers.
- `test_gen` contains all of Roc's code generation tests. See [README.md](./compiler/test_gen/README.md) for more information.
- `test_mono` Tests Roc's generation of the mono intermediate representation.
- `test_mono_macros` Macros for use in test_mono
- `roc_types` Various representations and utilities for dealing with types in the Roc compiler.
- `roc_unify` Implements Roc's unification algorithm, the heartstone of Roc's type inference.

## `docs/` - `roc_docs`

Generates html documentation from Roc files.
Used for [roc-lang.org/builtins/Num](https://www.roc-lang.org/builtins/Num).

## `docs_cli/` - `roc_docs_cli` library and `roc-docs` binary

Provides a binary that is only used for static build servers.

## `editor/` - `roc_editor`

For editing Roc files (Work In Progress). See [README.md](./editor/README.md) for more information.

## `error_macros/` - `roc_error_macros`

Provides macros for consistent reporting of errors in Roc's rust code.

## `glue/` - `roc_glue`

The `roc_glue` crate generates rust code to connect a Roc app with a rust platform. This tool is not necessary for writing a platform in another language, however, it's an added convenience for rust platform devs.

## `highlight/` - `roc_highlight`

Provides syntax highlighting for the editor by transforming a string to markup nodes.

## `linker/` - `roc_linker`

Surgical linker that links platforms to Roc applications.
We created our own linker for speed and because regular linkers add a lot of problems and complexity. Because we want roc to manage the build system and final linking of the executable, it is significantly less practical to use a regular linker. This can be seen in how many arbitrary libraries we link with the legacy linker because we have no idea what libraries the application is actually using.
See [README.md](./linker/README.md) for more information.

## `repl_cli/` - `roc_repl_cli`

Command Line Interface(CLI) functionality for the Read-Evaluate-Print-Loop (REPL).

## `repl_eval/` - `roc_repl_eval`

Provides the functionality for the REPL to evaluate Roc expressions.

## `repl_expect/` - `roc_repl_expect`

Supports evaluating `expect` and printing contextual information when they fail.

## `repl_test/` - `repl_test`

Tests the roc REPL.

## `repl_wasm/` - `roc_repl_wasm`

Provides a build of the REPL for the Roc website using WebAssembly. See [README.md](./repl_wasm/README.md) for more information.

## `reporting/` - `roc_reporting`

Responsible for generating warning and error messages.

## `roc_std/` - `roc_std`

Provides Rust representations of Roc data structures.

## `test_utils/` - `roc_test_utils`

Provides testing utility functions for use throughout the Rust code base.

## `tracing/` - `roc_tracing`

Provides tracing utility functions for various executable entry points.

## `utils/` - `roc_utils`

Provides utility functions used all over the code base.

## `vendor/`

These are files that were originally obtained somewhere else (e.g. crates.io) but which we needed to fork for some Roc-specific reason. See [README.md](./vendor/README.md) for more information.

## `wasi-libc-sys/` - `wasi_libc_sys`

Provides a Rust wrapper for the WebAssembly test platform built on libc and is primarily used for testing purposes.