catala/tests

Catala test suite

This folder contains Catala source files designed to test the features of the language.

It uses make to launch tests and compare the test terminal output with an expected output.

When you create a new test, please register it in the Makefile following the other examples. Expected outputs are stored using the convention <name_of_test>.catala.<name_of_scope>.out in the corresponding test folder.

For both workflows: use CATALA_OPTS="..." make ... to pass in Catala compiler options when debugging.

Workflow for adding new tests

1. Create a new test file in foo/bar.catala (pick the right directory and an informative name for your test)
2. Write your test, and pick a toplevel scope A to run.
3. From this directory, launch make foo/bar.catala.A.run to get the output of your test.
4. When you're happy with the output, launch make foo/bar.catala.A.in. This will record the content of the output of your test into a file.
5. Check that your test pass with make foo/bar.catala.A.out.
6. That's it, you've added a new test for the Catala language!

Workflow for fixing regressions

1. Run make, if a test fails you should see something like [FAIL foo/bar.catala.A].
2. Compare the computed and expected output with make foo/bar.catala.A.out.
3. Debug the compiler and/or the test, running make foo/bar.catala.A.run periodically to check the output of Catala on the test case.
4. When you're finished debugging, record the new test output with make foo/bar.catala.A.in.
5. Re-reun make to check that everything passes.
6. That's it, you've fixed the Catala test suite to adapt for changes in the language.