mirror of
https://github.com/AleoHQ/leo.git
synced 2024-12-01 02:08:31 +03:00
added doc
This commit is contained in:
parent
5d018f4ad2
commit
b20f8a0369
150
tests/README.md
Normal file
150
tests/README.md
Normal file
@ -0,0 +1,150 @@
|
||||
# Leo Test Framework
|
||||
|
||||
This directory includes Leo code samples, which are parsed and used as tests by test-framework.
|
||||
|
||||
## Structure
|
||||
|
||||
Currently, test framework covers only two areas: compiler and parser, tests for both destinations are placed in
|
||||
matching folders. Third folder - expectations - contains results of test execution which are saved in git and then
|
||||
compared to test output.
|
||||
|
||||
## Test Structure
|
||||
|
||||
Tests can be placed either in `compiler/` or `parser/` directories. Each test is a Leo file with correct (or intentionally
|
||||
incorrect) Leo code. What makes Leo file a test is block comment at the top of the file:
|
||||
|
||||
```
|
||||
/*
|
||||
namespace: Parse
|
||||
expectation: Pass
|
||||
*/
|
||||
|
||||
circuit X {
|
||||
x: u32,
|
||||
y: u32,
|
||||
}
|
||||
```
|
||||
|
||||
This comment contains YAML structure with set of mandatory and optional fields.
|
||||
|
||||
## Test Expectations
|
||||
|
||||
After first run of the tests, test expectations will be autogenerated and placed under `tests/expectations` directory.
|
||||
They will contain results of exection in detail (for example, in Compiler tests they include number of constraints and
|
||||
output registers).
|
||||
|
||||
During next test runs, results of each test are compared to stored expectations, and if stored expectations (say, number
|
||||
of constaints in Pedersen Hash example) don't match actual results, error will be through and test won't pass. Of course,
|
||||
there are two possible scenarios:
|
||||
|
||||
1. If test has failed because logic was changed intentionally, then expectations need to be deleted. New ones will be
|
||||
generated instead. And commit or PR should contain changes to expectations as well as to tests or code.
|
||||
2. If test should pass, then expectations should not be changed or removed.
|
||||
|
||||
## Test Configuration
|
||||
|
||||
Here is the list of all possible configuration options for compiler and parser tests.
|
||||
|
||||
#### namespace
|
||||
|
||||
```
|
||||
- Mandatory: yes
|
||||
- Namespace: all
|
||||
- Values: Compile / Parser
|
||||
```
|
||||
|
||||
Only two values are supported: `Parser` and `Compile`, the former is meant to be a parser test, the latter
|
||||
is, obviously, compiler test. Mind that it's `Compile` and NOT ~Compiler~.
|
||||
|
||||
In Parser namespace there are additional possible values to this field: `ParseStatement`, `ParseExpression` and `Token`.
|
||||
Each one of them allows testing Leo parser on different levels - lexer tokens or just expressions/statements.
|
||||
|
||||
Compiler tests always include complete Leo programs.
|
||||
|
||||
### expectation
|
||||
|
||||
```
|
||||
- Mandatory: yes
|
||||
- Namespace: all
|
||||
- Values: Pass / Fail
|
||||
```
|
||||
|
||||
This setting indicates whether given code is supposed to be run successfuly or we expect failure. Then, if test
|
||||
was marked as `Pass` and it actually failed, you'll know that something went wrong and test or compiler/parser need
|
||||
fixing.
|
||||
|
||||
### input_file (Compile)
|
||||
|
||||
```
|
||||
- Mandatory: no
|
||||
- Namespace: Compile
|
||||
- Values: <input file path>, ...
|
||||
```
|
||||
|
||||
This setting allows using one or multiple input files for Leo program. Program will then be run with every provided input.
|
||||
See this example:
|
||||
|
||||
```
|
||||
/*
|
||||
namespace: Compile
|
||||
expectation: Pass
|
||||
input_file:
|
||||
- inputs/a_0.in
|
||||
- inputs/a_1.in
|
||||
*/
|
||||
|
||||
function main(a: u32) {}
|
||||
```
|
||||
|
||||
### inputs (Compile)
|
||||
|
||||
```
|
||||
- Mandatory: no
|
||||
- Namespace: Compile
|
||||
- Values: <input file contents>, ...
|
||||
```
|
||||
|
||||
With this setting you can specify inputs right in the test description. It is useful for not creating too many files for
|
||||
each single case.
|
||||
|
||||
```
|
||||
/*
|
||||
namespace: Compile
|
||||
expectation: Pass
|
||||
inputs:
|
||||
- first.in: |
|
||||
[main]
|
||||
a: u32 = 100;
|
||||
|
||||
[registers]
|
||||
r0: bool = false;
|
||||
# - second.in: | ....
|
||||
*/
|
||||
|
||||
function main(a: u32) -> bool {
|
||||
return a == 100;
|
||||
}
|
||||
```
|
||||
|
||||
### state_file (Compile)
|
||||
|
||||
```
|
||||
- Mandatory: no
|
||||
- Namespace: Compile
|
||||
- Values: <path to state file>
|
||||
```
|
||||
|
||||
This setting allows you to specify state file, which can be accessed in Leo program:
|
||||
|
||||
```
|
||||
/*
|
||||
namespace: Compile
|
||||
expectation: Pass
|
||||
state_file: input/basic.state
|
||||
input_file: input/basic.in
|
||||
*/
|
||||
|
||||
function main(a: bool) -> bool {
|
||||
return a == input.registers.b;
|
||||
}
|
||||
```
|
Loading…
Reference in New Issue
Block a user