catala/tests
2020-12-30 12:02:09 +01:00
..
test_array Various small fixings 2020-12-30 12:02:09 +01:00
test_bool Array type is beginning to work 2020-12-29 22:26:10 +01:00
test_date Added assertions, starting to execute section132 2020-12-10 18:11:43 +01:00
test_dec Implemented all aggregate desugaring 2020-12-29 23:43:00 +01:00
test_default Array type is beginning to work 2020-12-29 22:26:10 +01:00
test_enum Added assertions, starting to execute section132 2020-12-10 18:11:43 +01:00
test_func Array type is beginning to work 2020-12-29 22:26:10 +01:00
test_money Debugging unification... 2020-12-30 01:02:04 +01:00
test_scope Array type is beginning to work 2020-12-29 22:26:10 +01:00
test_struct Added assertions, starting to execute section132 2020-12-10 18:11:43 +01:00
Makefile Better test suite terminal output 2020-12-21 16:15:42 +01:00
README.md Improved readmes 2020-12-14 10:59:15 +01:00

Catala test suite

This folder contains Catala source files designed to test the features of the language.

It uses make pass_tests to launch tests and compare the test terminal output with an expected output.

When you create a new test, please register it in the Makefile following the other examples. Expected outputs are stored using the convention <name_of_test>.catala.<name_of_scope>.out in the corresponding test folder.

For both workflows: use CATALA_OPTS="..." make ... to pass in Catala compiler options when debugging.

Workflow for adding new tests

  1. Create a new test file in foo/bar.catala (pick the right directory and an informative name for your test)
  2. Write your test, and pick a toplevel scope A to run.
  3. From this directory, launch make foo/bar.catala.A.run to get the output of your test.
  4. When you're happy with the output, launch make foo/bar.catala.A.in. This will record the content of the output of your test into a file.
  5. Check that your test pass with make foo/bar.catala.A.out.
  6. That's it, you've added a new test for the Catala language!

Workflow for fixing regressions

  1. Run make, if a test fails you should see something like [FAIL foo/bar.catala.A].
  2. Compare the computed and expected output with make foo/bar.catala.A.out.
  3. Debug the compiler and/or the test, running make foo/bar.catala.A.run periodically to check the output of Catala on the test case.
  4. When you're finished debugging, record the new test output with make foo/bar.catala.A.in.
  5. Re-reun make to check that everything passes.
  6. That's it, you've fixed the Catala test suite to adapt for changes in the language.

If a compiler change causes a lot of regressions (error message formatting changes for instance), you can mass-reset the expected the outputs with make reset_tests. Caution: use at your own risk, regressions should be fixed one by one in general.