mirror of
https://github.com/CatalaLang/catala.git
synced 2024-09-20 00:41:05 +03:00
9008a7f0e6
Follow-up of #287, #266 and #165. Time spent Pair programming sessions Before 2022-07-11: 50h (50 h for each person of the pair programming duo) Refactoring sessions Before 2022-07-11: 24 h 2022-07-14: 3 h Legal research sessions Before 2022-07-11: 21,5 h Testing and debugging Before 2022-07-11: 13,5 h 2022-07-11: 3 h with Denis 2022-07-13: 2 h with Denis 2022-07-14: 1 h with Denis 2022-07-16: 2 h with Denis 2022-07-19: 2 h with Denis 2022-07-21: 2 h with Denis 2022-08-11: 6 h with Denis 2022-08-15: 4 h with Denis 2022-08-16: 2 h with Denis UI and form 2022-08-09: 8 h with Denis 2022-08-10: 8 h with Denis 2022-08-15: 2 h with Denis 2022-08-16: 2 h with Denis 2022-08-17: 6 h with Denis 2022-08-18: 4 h with Denis |
||
|---|---|---|
| .. | ||
| test_arithmetic | ||
| test_array | ||
| test_bool | ||
| test_date | ||
| test_dec/good | ||
| test_default/bad | ||
| test_enum | ||
| test_exception | ||
| test_func | ||
| test_io | ||
| test_literate/good | ||
| test_metadata/good | ||
| test_money | ||
| test_name_resolution/good | ||
| test_proof | ||
| test_scope | ||
| test_struct | ||
| test_variable_state | ||
| Makefile | ||
| README.md | ||
Catala test suite
This folder contains Catala source files designed to test the features of the language.
Tests are declared inside plain Catala files with the following format:
```catala-test {id="TEST-IDENT"}
catala ARGS
```
The output of running catala with the given ARGS for a foo.catala_en file is
expected to be in an output/foo.catala_en.TEST-IDENT file (use
CLERK_OPTS=--reset to (re-)generate it).
You can use CATALA_OPTS="..." make ... to pass in Catala compiler options when
debugging.
Workflow for adding new tests
- Create a new test file in
foo/{good,bad}/bar.catala_<language>(pick the right directory and an informative name for your test) - Write your test, and pick a toplevel scope
Ato run. - Add the following section to your file:
```catala-test {id="A.Interpret"} catala Interpret -s A ``` - Run
CLERK_OPTS=--reset make tests/foo/{good,bad}/bar.catala_<language>from the root of the Catala repository. This will record the content of the output of your test into the target file. - Check that the expected output is present in
tests/foo/{good,bad}/output/bar.catala_<language>.A.Interpret - Don't forget to
git addboth the test file and the output file.
Workflow for fixing regressions
- Run
make test_suitefrom the root of the Catala repository, if a test fails you should see something like[ERROR] Test failed: foo/{good,bad}/output/bar.catala_<language>.A.Interpret]. - Compare the computed and expected output with
make tests/foo/{good,bad}/bar.catala_<language>from the root of the Catala repository. Debug the compiler and the test and repeat. - When you're finished debugging, if you are positive that a change in the test
output is legitimate, record the new outputs with
CLERK_OPTS=--reset make tests/foo/{good,bad}/bar.catala_<language>. - Re-run
make test_suitefrom the root of the Catala repository to check that everything passes. - Run
git diffto double-check your changes to the test outputs are expected. If necessary, justify them in your commit message.
If a compiler change causes a lot of regressions (error message formatting changes
for instance), you can mass-reset the expected the outputs with CLERK_OPTS=--reset make test_suite.
Caution: It's your responsability to check all the changes before committing them.