mirror of https://github.com/CatalaLang/catala.git synced 2024-09-20 00:41:05 +03:00

History

Louis Gesbert 8c1696d0ff Inline tests: cleanup trailing whitespace		2022-09-23 16:56:21 +02:00
..
test_arithmetic	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_array	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_bool	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_date	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_dec/good	Turn all existing tests to inline tests	2022-09-23 14:45:10 +02:00
test_default/bad	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_enum	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_exception	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_func	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_io	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_literate/good	Add test return codes	2022-09-23 14:50:02 +02:00
test_metadata/good	Turn all existing tests to inline tests	2022-09-23 14:45:10 +02:00
test_money	Inline tests: cleanup trailing whitespace	2022-09-23 16:56:21 +02:00
test_name_resolution/good	Turn all existing tests to inline tests	2022-09-23 14:45:10 +02:00
test_proof	Turn all existing tests to inline tests	2022-09-23 14:45:10 +02:00
test_scope	Add test return codes	2022-09-23 14:50:02 +02:00
test_struct	Add test return codes	2022-09-23 14:50:02 +02:00
test_typing/bad	Add test return codes	2022-09-23 14:50:02 +02:00
test_variable_state	Add test return codes	2022-09-23 14:50:02 +02:00
Makefile	Fix documentation and test invocation	2022-08-30 13:48:20 +02:00
README.md	Clerk tests: update README	2022-09-23 16:41:22 +02:00

README.md

Catala test suite

This folder contains Catala source files designed to test the features of the language.

Tests are declared inside plain Catala files with the following format:

```catala-test-inline
$ catala ARGS
... output from the catala command ...
```

Workflow for adding new tests

Create a new test file in foo/{good,bad}/bar.catala_<language> (pick the right directory and an informative name for your test)
Write your test, and pick a toplevel scope A to run.

Add the following section to your file:

```catala-test-inline
$ catala Interpret -s A
```

Run make tests/foo/{good,bad}/bar.catala_<language> CLERK_OPTS=--reset from the root of the Catala repository. This will update the test with the actual output of the catala command.
Don't forget to git add the test file.

Workflow for fixing regressions

Run make test_suite from the root of the Catala repository, if a test fails you should see something like FAILED: foo/{good,bad}/bar.catala_<language>.out] followed by the diff between the expected output and the current output of the command.
Debug the compiler and the test and repeat. Run make tests/foo/{good,bad}/bar.catala_<language> to check the test again.
When you're finished debugging, if you are positive that a change in the test output is legitimate, record the new outputs with make tests/foo/{good,bad}/bar.catala_<language> CLERK_OPTS=--reset.
Re-run make test_suite from the root of the Catala repository to check that everything passes.
Run git diff to double-check your changes to the test outputs are expected. If necessary, justify them in your commit message.

If a compiler change causes a lot of regressions (error message formatting changes for instance), you can mass-reset the expected outputs with make test_suite CLERK_OPTS=--reset. git diff will then allow to check all the changes at once. Caution: It's your responsability to check all the changes before committing them.