From b20f8a0369cc6e480b1974ce6f9de11238f6d0b1 Mon Sep 17 00:00:00 2001
From: damirka <damirka.ru@gmail.com>
Date: Tue, 11 May 2021 21:16:55 +0300
Subject: [PATCH] added doc

---
 tests/README.md | 150 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 150 insertions(+)
 create mode 100644 tests/README.md

diff --git a/tests/README.md b/tests/README.md
new file mode 100644
index 0000000000..04c9a3be29
--- /dev/null
+++ b/tests/README.md
@@ -0,0 +1,150 @@
+# Leo Test Framework
+
+This directory includes Leo code samples, which are parsed and used as tests by test-framework.
+
+## Structure
+
+Currently, test framework covers only two areas: compiler and parser, tests for both destinations are placed in
+matching folders. Third folder - expectations - contains results of test execution which are saved in git and then 
+compared to test output.
+
+## Test Structure
+
+Tests can be placed either in `compiler/` or `parser/` directories. Each test is a Leo file with correct (or intentionally 
+incorrect) Leo code. What makes Leo file a test is block comment at the top of the file:
+
+```
+/*
+namespace: Parse
+expectation: Pass
+*/
+
+circuit X {
+    x: u32,
+    y: u32,
+}
+```
+
+This comment contains YAML structure with set of mandatory and optional fields.
+
+## Test Expectations
+
+After first run of the tests, test expectations will be autogenerated and placed under `tests/expectations` directory.
+They will contain results of exection in detail (for example, in Compiler tests they include number of constraints and 
+output registers).
+
+During next test runs, results of each test are compared to stored expectations, and if stored expectations (say, number
+of constaints in Pedersen Hash example) don't match actual results, error will be through and test won't pass. Of course,
+there are two possible scenarios:
+
+1. If test has failed because logic was changed intentionally, then expectations need to be deleted. New ones will be
+generated instead. And commit or PR should contain changes to expectations as well as to tests or code.
+2. If test should pass, then expectations should not be changed or removed.
+
+## Test Configuration
+
+Here is the list of all possible configuration options for compiler and parser tests.
+
+#### namespace
+
+```
+- Mandatory: yes
+- Namespace: all
+- Values: Compile / Parser
+```
+
+Only two values are supported: `Parser` and `Compile`, the former is meant to be a parser test, the latter 
+is, obviously, compiler test. Mind that it's `Compile` and NOT ~Compiler~. 
+
+In Parser namespace there are additional possible values to this field: `ParseStatement`, `ParseExpression` and `Token`. 
+Each one of them allows testing Leo parser on different levels - lexer tokens or just expressions/statements.
+
+Compiler tests always include complete Leo programs.
+
+### expectation
+
+```
+- Mandatory: yes
+- Namespace: all
+- Values: Pass / Fail
+```
+
+This setting indicates whether given code is supposed to be run successfuly or we expect failure. Then, if test
+was marked as `Pass` and it actually failed, you'll know that something went wrong and test or compiler/parser need
+fixing.
+
+### input_file (Compile)
+
+```
+- Mandatory: no
+- Namespace: Compile
+- Values: <input file path>, ...
+```
+
+This setting allows using one or multiple input files for Leo program. Program will then be run with every provided input.
+See this example:
+
+```
+/*
+namespace: Compile
+expectation: Pass
+input_file: 
+ - inputs/a_0.in
+ - inputs/a_1.in
+*/
+
+function main(a: u32) {}
+```
+
+### inputs (Compile)
+
+```
+- Mandatory: no
+- Namespace: Compile
+- Values: <input file contents>, ...
+```
+
+With this setting you can specify inputs right in the test description. It is useful for not creating too many files for
+each single case.
+
+```
+/*
+namespace: Compile
+expectation: Pass
+inputs: 
+ - first.in: |
+    [main]
+    a: u32 = 100;
+    
+    [registers]
+    r0: bool = false;
+# - second.in: | ....
+*/
+
+function main(a: u32) -> bool {
+    return a == 100;
+}
+```
+
+### state_file (Compile)
+
+```
+- Mandatory: no
+- Namespace: Compile
+- Values: <path to state file>
+```
+
+This setting allows you to specify state file, which can be accessed in Leo program:
+
+```
+/*
+namespace: Compile
+expectation: Pass
+state_file: input/basic.state
+input_file: input/basic.in
+*/
+
+function main(a: bool) -> bool {
+    return a == input.registers.b;
+}
+```