CatalaLang/catala
1
1
Fork 0
mirror of https://github.com/CatalaLang/catala.git synced 2024-09-20 00:41:05 +03:00
Code Issues Projects Releases Wiki Activity

docs(test): update the README.md

Browse Source
This commit is contained in:
EmileRolley 2021-05-26 21:57:51 +02:00
parent e4d1d5e2e6
commit ec43e82d09
1 changed files with 15 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
tests/README.md
View File

@ -1,44 +1,43 @@
# Catala test suite
This folder contains Catala source files designed to test the features of the
This folder contains Catala source files designed to test the features of the
language.
It uses `make pass_tests` to launch tests and compare the test terminal output
with an expected output.
Expected outputs are stored using the convention
`<name_of_test>.catala.<name_of_scope>.out` in the corresponding test output folder.
Expected outputs are stored using the convention
`<name_of_test>.catala_<language>.<name_of_scope>.out` in the corresponding test output folder.
For both workflows: use `CATALA_OPTS="..." make ...` to pass in Catala compiler
options when debugging.
## Workflow for adding new tests
1. Create a new test file in `foo/{good,bad}/bar.catala` (pick the right directory and
1. Create a new test file in `foo/{good,bad}/bar.catala_<language>` (pick the right directory and
an informative name for your test)
2. Write your test, and pick a toplevel scope `A` to run.
3. From this directory, launch `make foo/{good,bad}/bar.catala.A.run` to get the output of
3. From this directory, launch `make foo/{good,bad}/bar.catala_<language>.A.run` to get the output of
your test.
4. When you're happy with the output, launch `make foo/{good,bad}/bar.catala.A.in`. This
4. When you're happy with the output, launch `make foo/{good,bad}/bar.catala_<language>.A.in`. This
will record the content of the output of your test into a file.
5. Check that your test pass with `make foo/{good,bad}/output/bar.catala.A.out`.
5. Check that your test pass with `make foo/{good,bad}/output/bar.catala_<language>.A.out`.
6. That's it, you've added a new test for the Catala language!
## Workflow for fixing regressions
1. Run `make`, if a test fails you should see something like
`[FAIL foo/{good,bad}/output/bar.catala.A]`.
2. Compare the computed and expected output with `make foo/{good,bad}/output/bar.catala.A.out`.
3. Debug the compiler and/or the test, running `make foo/{good,bad}/bar.catala.A.run`
1. Run `make`, if a test fails you should see something like
`[FAIL foo/{good,bad}/output/bar.catala_<language>.A]`.
2. Compare the computed and expected output with `make foo/{good,bad}/output/bar.catala_<language>.A.out`.
3. Debug the compiler and/or the test, running `make foo/{good,bad}/bar.catala_<language>.A.run`
periodically to check the output of Catala on the test case.
4. When you're finished debugging, record the new test output with
`make foo/{good,bad}/bar.catala.A.in`.
4. When you're finished debugging, record the new test output with
`make foo/{good,bad}/bar.catala_<language>.A.in`.
5. Re-run `make` to check that everything passes.
6. That's it, you've fixed the Catala test suite to adapt for changes in the
6. That's it, you've fixed the Catala test suite to adapt for changes in the
language.
If a compiler change causes a lot of regressions (error message formatting changes
for instance), you can mass-reset the expected the outputs with `make reset_tests`.
**Caution**: use at your own risk, regressions should be fixed one by one in
**Caution**: use at your own risk, regressions should be fixed one by one in
general.