Not much there at the moment, but being able to specify the include directories
is already pretty useful to run clerk directly e.g. on `catala-examples`.
(you had to explicitely specify variable `CATALA_INCLUDE`, the `-I` flags or to
go through `make` without that)
This is a proper replacement for the previous shell-based placeholder hack.
Here is a summary:
- `clerk runtest` (normally run by ninja) is much extended:
* besides generating the test@out file, it checks individual tests for success
and can write a report file containing their status, and the positions for
their (expected/current) outputs (this uses `Marshal`)
* it now handles out-tests directly in addition to inline-tests, for which
it generates the separate output file ; they are included in the report
- ninja is now tasked with building all the test reports (which shouldn't fail);
for directories, individual reports are concatenated (as before).
Removing intermediate report rules, and out-test rules means that the ninja
file is much simplified.
- then, clerk takes back control, reads the final reports and formats them in a
user-friendly way. Printing the reports may imply running `diff` internally.
In particular, the commands to easily reproduce each test are provided.
Resetting the test results if required is also done directly by clerk, at this
stage.
A few switches are available to customise the output, but I am waiting for some
feedback before deciding what to make available from the CLI.
The `clerk report` command is available to manually explore test reports, but
normally the processing is done directly at the end of `clerk test` (i.e. ninja
will no longer call that command)
Runtimes for the various backends are expected to be made available from their
own ecosystem. However, for convenience and to help with development
settings (where the runtime might change), as part of installing catala they are
put, in source form, into `<prefix>/lib/catala/runtime_LANG`.
When using a dev version of Catala, and using Python, one would then just have
to do `pip install <prefix>/lib/catala/runtime_python` within their venv to be
able to run their python programs.
* Obsolete code for included tests has been removed
* The engine uses a proper lexer and is much simplified
* An inline test in the middle of the file now only "sees" the file up to that
point. This fixes an issue where we had spurious errors when a type error was
added at the end of a file, and it would pop up in tests before it. This makes
files including many tests much more practical.
* diffing and resetting the tests has been reintroduced (done at the moment in
Ninja, but for more control (count number of failed tests, etc.) we could put it
back into Clerk at some point
* The Catala CLI can now take an input from stdin (with the possibility to link
a (possibly fake) on-disk file for error reporting and file locations ; this
is useful for running tests)
We have an obvious limitation: static builds won't support plugins. We
could resort to the classic method and do normal builds but on an old
distrib to dodge compatibility issues.
That would still be compatible with statically linking the more specific
libs, like e.g. z3.
