mirror of
https://github.com/AleoHQ/leo.git
synced 2024-12-22 17:11:42 +03:00
151 lines
3.8 KiB
Markdown
151 lines
3.8 KiB
Markdown
# Leo Test Framework
|
|
|
|
This directory includes Leo code samples, which are parsed and used as tests by test-framework.
|
|
|
|
## Structure
|
|
|
|
Currently, test framework covers only two areas: compiler and parser, tests for both destinations are placed in
|
|
matching folders. Third folder - expectations - contains results of test execution which are saved in git and then
|
|
compared to test output.
|
|
|
|
## Test Structure
|
|
|
|
Tests can be placed either in `compiler/` or `parser/` directories. Each test is a Leo file with correct (or intentionally
|
|
incorrect) Leo code. What makes Leo file a test is block comment at the top of the file:
|
|
|
|
```
|
|
/*
|
|
namespace: Parse
|
|
expectation: Pass
|
|
*/
|
|
|
|
circuit X {
|
|
x: u32,
|
|
y: u32,
|
|
}
|
|
```
|
|
|
|
This comment contains YAML structure with set of mandatory and optional fields.
|
|
|
|
## Test Expectations
|
|
|
|
After first run of the tests, test expectations will be autogenerated and placed under `tests/expectations` directory.
|
|
They will contain results of exection in detail (for example, in Compiler tests they include number of constraints and
|
|
output registers).
|
|
|
|
During next test runs, results of each test are compared to stored expectations, and if stored expectations (say, number
|
|
of constaints in Pedersen Hash example) don't match actual results, error will be through and test won't pass. Of course,
|
|
there are two possible scenarios:
|
|
|
|
1. If test has failed because logic was changed intentionally, then expectations need to be deleted. New ones will be
|
|
generated instead. And commit or PR should contain changes to expectations as well as to tests or code.
|
|
2. If test should pass, then expectations should not be changed or removed.
|
|
|
|
## Test Configuration
|
|
|
|
Here is the list of all possible configuration options for compiler and parser tests.
|
|
|
|
#### namespace
|
|
|
|
```
|
|
- Mandatory: yes
|
|
- Namespace: all
|
|
- Values: Compile / Parser
|
|
```
|
|
|
|
Only two values are supported: `Parser` and `Compile`, the former is meant to be a parser test, the latter
|
|
is, obviously, compiler test. Mind that it's `Compile` and NOT ~Compiler~.
|
|
|
|
In Parser namespace there are additional possible values to this field: `ParseStatement`, `ParseExpression` and `Token`.
|
|
Each one of them allows testing Leo parser on different levels - lexer tokens or just expressions/statements.
|
|
|
|
Compiler tests always include complete Leo programs.
|
|
|
|
### expectation
|
|
|
|
```
|
|
- Mandatory: yes
|
|
- Namespace: all
|
|
- Values: Pass / Fail
|
|
```
|
|
|
|
This setting indicates whether given code is supposed to be run successfuly or we expect failure. Then, if test
|
|
was marked as `Pass` and it actually failed, you'll know that something went wrong and test or compiler/parser need
|
|
fixing.
|
|
|
|
### input_file (Compile)
|
|
|
|
```
|
|
- Mandatory: no
|
|
- Namespace: Compile
|
|
- Values: <input file path>, ...
|
|
```
|
|
|
|
This setting allows using one or multiple input files for Leo program. Program will then be run with every provided input.
|
|
See this example:
|
|
|
|
```
|
|
/*
|
|
namespace: Compile
|
|
expectation: Pass
|
|
input_file:
|
|
- inputs/a_0.in
|
|
- inputs/a_1.in
|
|
*/
|
|
|
|
function main(a: u32) {}
|
|
```
|
|
|
|
### inputs (Compile)
|
|
|
|
```
|
|
- Mandatory: no
|
|
- Namespace: Compile
|
|
- Values: <input file contents>, ...
|
|
```
|
|
|
|
With this setting you can specify inputs right in the test description. It is useful for not creating too many files for
|
|
each single case.
|
|
|
|
```
|
|
/*
|
|
namespace: Compile
|
|
expectation: Pass
|
|
inputs:
|
|
- first.in: |
|
|
[main]
|
|
a: u32 = 100;
|
|
|
|
[registers]
|
|
r0: bool = false;
|
|
# - second.in: | ....
|
|
*/
|
|
|
|
function main(a: u32) -> bool {
|
|
return a == 100;
|
|
}
|
|
```
|
|
|
|
### state_file (Compile)
|
|
|
|
```
|
|
- Mandatory: no
|
|
- Namespace: Compile
|
|
- Values: <path to state file>
|
|
```
|
|
|
|
This setting allows you to specify state file, which can be accessed in Leo program:
|
|
|
|
```
|
|
/*
|
|
namespace: Compile
|
|
expectation: Pass
|
|
state_file: input/basic.state
|
|
input_file: input/basic.in
|
|
*/
|
|
|
|
function main(a: bool) -> bool {
|
|
return a == input.registers.b;
|
|
}
|
|
```
|