CatalaLang/catala
1
1
Fork 0
mirror of https://github.com/CatalaLang/catala.git synced 2024-11-08 07:51:43 +03:00
Code Issues Projects Releases Wiki Activity
17592d060e
catala/tests
History
Louis Gesbert 05e3205fc9 Parser: priority tweaks 
- makes sums and mults left-associative instead of right-associative (see the
  change in the tests!)
- uses higher priority for unary operators
2022-05-11 19:26:34 +02:00
..
test_arithmetic Parser: priority tweaks 2022-05-11 19:26:34 +02:00
test_array Update syntax highlighting and test suite [skip ci] 2022-02-07 12:04:48 +01:00
test_bool Tests now passing except for a few position and printing issues [skip ci] 2022-04-04 12:25:00 +02:00
test_date Update syntax highlighting and test suite [skip ci] 2022-02-07 12:04:48 +01:00
test_dec/good Parser: priority tweaks 2022-05-11 19:26:34 +02:00
test_default/bad fix typo 2022-04-07 22:43:05 +02:00
test_enum Merge branch 'master' into c_backend 2022-04-04 18:02:33 +02:00
test_exception fix typo 2022-04-07 22:43:05 +02:00
test_func fix typo 2022-04-07 22:43:05 +02:00
test_io Tests now passing except for a few position and printing issues [skip ci] 2022-04-04 12:25:00 +02:00
test_literate/good Update syntax highlighting and test suite [skip ci] 2022-02-07 12:04:48 +01:00
test_metadata/good Update syntax highlighting and test suite [skip ci] 2022-02-07 12:04:48 +01:00
test_money Rename test 2022-05-05 17:34:48 +02:00
test_name_resolution/good Update syntax highlighting and test suite [skip ci] 2022-02-07 12:04:48 +01:00
test_proof Merge branch 'master' into c_backend 2022-04-04 18:02:33 +02:00
test_scope fix typo 2022-04-07 22:43:05 +02:00
test_struct Merge branch 'master' into c_backend 2022-04-04 18:02:33 +02:00
test_variable_state Tests now passing except for a few position and printing issues [skip ci] 2022-04-04 12:25:00 +02:00
Makefile Pass some make flags to Clerk 2022-04-14 11:47:18 +02:00
README.md Test folders, nice doc and messages 2022-01-11 12:51:34 +01:00

README.md

Catala test suite

This folder contains Catala source files designed to test the features of the language.

Expected outputs are stored using the convention <name_of_test>.catala_<language>.<name_of_scope>.Interpret in the corresponding test output folder.

For both workflows: use CATALA_OPTS="..." make ... to pass in Catala compiler options when debugging.

Workflow for adding new tests

  1. Create a new test file in foo/{good,bad}/bar.catala_<language> (pick the right directory and an informative name for your test)
  2. Write your test, and pick a toplevel scope A to run.
  3. From this directory, create mkdir -p foo/{good,bad}/output and the empty expected output file touch foo/{good,bad}/output/bar.catala_<language>.A.Interpret.
  4. From the root of the Catala repository, Run make -C tests foo/{good,bad}/bar.catala_<language> to get the output of your test (compared with the empty file). The failing test will give you the command to use to reproduce the fail from the tests directory.
  5. When you're happy with the output, launch CLERK_OPTS=--reset make -C tests foo/{good,bad}/bar.catala_<language> from the root of the Catala repository. This will record the content of the output of your test into a file.
  6. Check that your test pass with make -C tests foo/{good,bad}/bar.catala_<language> from the root of the Catala repository.
  7. That's it, you've added a new test for the Catala language!

Workflow for fixing regressions

  1. Run make test_suite from the root of the Catala repository, if a test fails you should see something like [ERROR] Test failed: foo/{good,bad}/output/bar.catala_<language>.A.Interpret].
  2. Compare the computed and expected output with make -C tests foo/{good,bad}/bar.catala_<language> from the root of the Catala repository.
  3. Debug the compiler and/or the test, running make -C tests foo/{good,bad}/bar.catala_<language> from the root of the Catala repository periodically to check the output of Catala on the test case.
  4. When you're finished debugging, record the new test output with CLERK_OPTS=--reset make foo/{good,bad}/bar.catala_<language>.
  5. Re-run make test_suite from the root of the Catala repository to check that everything passes.
  6. That's it, you've fixed the Catala test suite to adapt for changes in the language.

If a compiler change causes a lot of regressions (error message formatting changes for instance), you can mass-reset the expected the outputs with CLERK_OPTS=--reset make test_suite. Caution: use at your own risk, regressions should be fixed one by one in general.