Note: we don't really detect the changes, maybe a hash could help; but at least
this avoids an ugly `Index out of bounds` in the case where the changes caused
a line to be shorter than expected.
This is a proper replacement for the previous shell-based placeholder hack.
Here is a summary:
- `clerk runtest` (normally run by ninja) is much extended:
* besides generating the test@out file, it checks individual tests for success
and can write a report file containing their status, and the positions for
their (expected/current) outputs (this uses `Marshal`)
* it now handles out-tests directly in addition to inline-tests, for which
it generates the separate output file ; they are included in the report
- ninja is now tasked with building all the test reports (which shouldn't fail);
for directories, individual reports are concatenated (as before).
Removing intermediate report rules, and out-test rules means that the ninja
file is much simplified.
- then, clerk takes back control, reads the final reports and formats them in a
user-friendly way. Printing the reports may imply running `diff` internally.
In particular, the commands to easily reproduce each test are provided.
Resetting the test results if required is also done directly by clerk, at this
stage.
A few switches are available to customise the output, but I am waiting for some
feedback before deciding what to make available from the CLI.
The `clerk report` command is available to manually explore test reports, but
normally the processing is done directly at the end of `clerk test` (i.e. ninja
will no longer call that command)
* Obsolete code for included tests has been removed
* The engine uses a proper lexer and is much simplified
* An inline test in the middle of the file now only "sees" the file up to that
point. This fixes an issue where we had spurious errors when a type error was
added at the end of a file, and it would pop up in tests before it. This makes
files including many tests much more practical.
* diffing and resetting the tests has been reintroduced (done at the moment in
Ninja, but for more control (count number of failed tests, etc.) we could put it
back into Clerk at some point
* The Catala CLI can now take an input from stdin (with the possibility to link
a (possibly fake) on-disk file for error reporting and file locations ; this
is useful for running tests)
- Use separate functions for successive passes in module `Driver.Passes`
- Use other functions for end results printing in module `Driver.Commands`
As a consequence, it is much more flexible to use by plugins or libs and we no
longer need the complex polymorphic variant parameter.
This patch leverages previous changes to use Cmdliner subcommands and
effectively specialises the flags of each Catala subcommand.
Other changes include:
- an attempt to normalise the generic options and reduce the number of global
references. Some are ok, like `debug` ; some would better be further cleaned up,
e.g. the ones used by Proof backend were moved to a `Proof.globals` module and
need discussion. The printer no longer relies on the global languages and prints
money amounts in an agnostic way.
- the plugin directory is automatically guessed and loaded even in dev setups.
Plugins are shown by the main `catala` command and listed in `catala --help`
- exception catching at the toplevel has been refactored a bit as well; return
codes are normalised to follow the manpage and avoid codes >= 128 that are
generally reserved for shells.
Update tests
This adds a few positions to the parser, and tweaks some others, vastly
improving the reporting of some errors (inconsistent functions definitions, but
also exceptions cycles, etc.)
