leo/docs/rfc/003-imports-stabilization.md

220 lines
7.0 KiB
Markdown
Raw Normal View History

2021-06-22 19:45:19 +03:00
# Leo RFC 003: Imports Stabilization
## Authors
- Max Bruce
- Collin Chin
- Alessandro Coglio
- Eric McCarthy
- Jon Pavlik
- Damir Shamanaev
- Damon Sicore
- Howard Wu
## Status
DRAFT
# Summary
2021-06-23 13:07:44 +03:00
This proposal aims to improve the import management system in Leo programs to
2021-06-30 22:00:48 +03:00
make program environment more reproducible, predictable and compatible. To achieve
that we suggest few changes to Leo CLI and Manifest:
2021-06-22 19:45:19 +03:00
2021-06-23 13:07:44 +03:00
- add a "dependencies" section to Leo Manifest and add a command to pull those dependencies;
2021-06-22 19:45:19 +03:00
- allow custom names for imports to manually resolve name conflicts;
2021-06-28 21:20:45 +03:00
- add "curve" and "proving system" sections to the Manifest;
2021-07-01 00:47:34 +03:00
- add "include" and "exclude" parameters for "proving system" and "curve";
2021-07-20 18:04:42 +03:00
- add a lock file which would store imported dependencies and their relations;
2021-06-22 19:45:19 +03:00
# Motivation
2021-06-28 21:20:45 +03:00
The current design of imports does not provide any guarantees on what's stored
in program imports and published with the program to Aleo Package Manager.
2021-07-01 19:14:06 +03:00
When a dependency is "added," it is stored inside imports folder, and it is possible
2021-06-28 21:20:45 +03:00
to manually edit and/or add packages in this folder.
2021-06-22 19:45:19 +03:00
2021-07-01 19:14:06 +03:00
Also, imports are stored under the package name which makes it impossible to import
2021-06-28 21:20:45 +03:00
two different packages with the same name.
2021-06-22 19:45:19 +03:00
2021-06-28 21:20:45 +03:00
Another important detail in the scope of this proposal is that in future Leo
programs will have the ability to be run with different proving systems
and curves, possibly creating incompatibility between programs written
2021-07-01 19:14:06 +03:00
for different proving systems or curves. To make a foundation for these features,
2021-06-28 21:20:45 +03:00
imports need to be managed with include/exclude lists for allowed (compatible)
proving systems and curves.
2021-06-22 19:45:19 +03:00
# Design
2021-06-30 22:00:48 +03:00
## Leo Manifest - target section
2021-07-01 19:14:06 +03:00
To lay the foundation for the future of the Leo ecosystem and start integrating
2021-06-30 22:00:48 +03:00
information about programs compatibility we suggest adding two new fields to
the new `[target]` section of the Leo Manifest: `proving_system` and `curve`.
2021-07-01 19:14:06 +03:00
Currently, the Leo compiler only supports `Groth16` for the proving system and `Bls12_377`
for the curve, they are meant to be default values in Leo Manifest.
2021-06-30 22:00:48 +03:00
```toml
[project]
name = "first"
version = "0.1.0"
description = "The first package"
license = "MIT"
[target]
curve = "Bls12_377"
proving_system = "Groth16"
```
These fields are meant to be used to determine whether imported program is
compatible to the original when support for different curves and proving systems
is added.
## Leo Manifest - dependencies
2021-06-22 19:45:19 +03:00
Dependencies section:
```toml
[dependencies]
name = { author = "author", package = "package", version = "version" }
2021-06-30 22:00:48 +03:00
# alternative way of adding dependency record
2021-06-22 19:45:19 +03:00
[dependencies.name]
author = "author"
package = "package"
version = "1.0"
```
2021-06-30 22:00:48 +03:00
### Parameters description
`name` field sets the name of the dependency in Leo code. That way we allow
developer to resolve collisions in import names manually. So, for example,
if a developer is adding `howard/silly-sudoku` package to his program, he
2021-06-30 22:14:36 +03:00
might define its in-code name as `sudoku` and import it with that name:
2021-06-30 22:00:48 +03:00
```ts
import sudoku;
```
`package`, `author` and `version` are package name, package author and
version respectively. They are already used as arguments in `leo add`
command, so these fields are already understood by the Leo developers.
2021-06-22 19:45:19 +03:00
## Leo CLI
To support updated Manifest new command should be added to Leo CLI.
```bash
# pull imports
2021-07-20 18:04:42 +03:00
leo fetch
2021-06-22 19:45:19 +03:00
```
## Imports Restructurization
One of the goals of proposed changes is to allow importing packages with the
2021-06-30 22:00:48 +03:00
same name but different authors. This has to be solved not only on the
language level but also on the level of storing program imports.
We suggest using set of all 3 possible program identifiers for import
folder name: `author-package@version`. Later it can be extended to
2021-07-01 19:14:06 +03:00
include hash for version, but having the inital set already solves name
2021-06-30 22:00:48 +03:00
collisions.
So, updated imports would look like:
```
leo-program
├── Leo.toml
├── README.md
├── imports
│ ├── author1-program@0.1.0
│ │ └── ...
│ ├── author2-program2@1.0.4
│ └── ...
├── inputs
│ └── ...
└── src
└── main.leo
```
2021-07-01 19:14:06 +03:00
This change would also affect the way imports are being processed on the ASG
level, and we'd need to add an imports map as an argument to the Leo compiler.
2021-06-30 22:14:36 +03:00
The Leo Manifest's dependencies sections needs to be parsed and passed as
a hashmap to the compiler:
2021-06-30 22:00:48 +03:00
```
first-program => author1-program@0.1.0
second-program => author2-program2@1.0.4
```
2021-06-22 19:45:19 +03:00
2021-07-20 18:04:42 +03:00
## Leo.lock
For imports map to be generated and read by the Leo binary and then by the Leo compiler,
2021-07-20 20:55:45 +03:00
a lock file needs to be created. Lock file should be generated by the `leo fetch` command,
which will pull the dependencies, process their manifests, and put the required information
2021-07-20 18:04:42 +03:00
to the file in the root directory of the program called `Leo.lock`.
Suggested structure of this file is similar to the Cargo.lock file:
```
[[package]]
name = "suit-mk2"
version = "0.2.0"
author = "ironman"
import_name = "suit-mk2"
[package.dependencies]
garbage = "ironman-suit@0.1.0"
[[package]]
name = "suit"
version = "0.1.0"
author = "ironman"
import_name = "garbage"
```
2021-07-20 20:55:45 +03:00
In the example above, you can see that all program dependencies are defined as an
array called `package`. Each of the dependencies contains main information about
it, including the `import_name` field which is the imported package's name in
2021-07-20 18:04:42 +03:00
the Leo program. Also, it stores relationships between these dependencies in the
field `dependencies`.
2021-07-20 20:55:45 +03:00
The format described here allows the Leo binary to form an imports map which can be
2021-07-20 18:04:42 +03:00
passed to the compiler.
2021-07-20 20:55:45 +03:00
It is important to note that Leo.lock file is created only when a package has dependencies.
For programs with no dependencies, a lock file is not required and not created.
2021-07-20 18:04:42 +03:00
2021-06-30 22:00:48 +03:00
## Recursive Dependencies
2021-06-22 19:45:19 +03:00
2021-06-30 22:00:48 +03:00
This improvement introduces recursive dependencies. To solve this case preemptively
2021-07-01 19:14:06 +03:00
Leo CLI needs to check the dependency tree and throw an error when a recursive dependency
2021-06-30 22:00:48 +03:00
is met. We suggest implementing simple dependency tree checking while fetching
2021-07-01 19:14:06 +03:00
imports - if imported dependency is met on a higher level - abort the execution.
2021-06-22 19:45:19 +03:00
2021-06-30 22:00:48 +03:00
Later this solution can be improved by building a lock file containing all the
information on program dependencies, and the file itself will have enough data
to track and prevent recursion.
2021-06-22 19:45:19 +03:00
# Drawbacks
2021-06-23 13:07:44 +03:00
This change might require the update of already published programs on Aleo PM due to
2021-06-22 19:45:19 +03:00
Leo Manifest change. However it is possible to implement it in a backward-compatible
way.
2021-06-30 22:00:48 +03:00
It also introduces the danger of having recursive dependencies, this problem is addressed in the Design section above.
2021-06-22 19:45:19 +03:00
# Effect on Ecosystem
Proposed improvement provides safety inside Leo programs and should not affect
2021-06-23 13:07:44 +03:00
ecosystem except for the tools which use Leo directly (such as Aleo Studio).
2021-06-22 19:45:19 +03:00
2021-06-23 13:07:44 +03:00
It is possible that some of the proposed features will open new features on Aleo PM.
2021-06-22 19:45:19 +03:00
# Alternatives
2021-06-23 13:07:44 +03:00
Another approach to the stated cases is to keep everything as we have now but change
2021-06-22 19:45:19 +03:00
the way programs are imported and stored and make names unique. Also, current
implementation provides some guarantees on import stablitity and consistency.