catala/compiler
Louis Gesbert 45b0feaf20
Generate tests reports from 'clerk test'
This is a proper replacement for the previous shell-based placeholder hack.

Here is a summary:

- `clerk runtest` (normally run by ninja) is much extended:
  * besides generating the test@out file, it checks individual tests for success
    and can write a report file containing their status, and the positions for
    their (expected/current) outputs (this uses `Marshal`)
  * it now handles out-tests directly in addition to inline-tests, for which
    it generates the separate output file ; they are included in the report

- ninja is now tasked with building all the test reports (which shouldn't fail);
  for directories, individual reports are concatenated (as before).
  Removing intermediate report rules, and out-test rules means that the ninja
  file is much simplified.

- then, clerk takes back control, reads the final reports and formats them in a
  user-friendly way. Printing the reports may imply running `diff` internally.
  In particular, the commands to easily reproduce each test are provided.
  Resetting the test results if required is also done directly by clerk, at this
  stage.

A few switches are available to customise the output, but I am waiting for some
feedback before deciding what to make available from the CLI.

The `clerk report` command is available to manually explore test reports, but
normally the processing is done directly at the end of `clerk test` (i.e. ninja
will no longer call that command)
2024-06-19 16:10:26 +02:00
..
catala_utils Generate tests reports from 'clerk test' 2024-06-19 16:10:26 +02:00
dcalc More precise positions for operators throughout 2024-04-30 16:35:08 +02:00
desugared Add multiple typing errors using delayed errors 2024-06-19 14:52:31 +02:00
lcalc Small code cleanups 2024-06-19 16:10:15 +02:00
literate Rewriting message calls to use the new intf 2024-04-10 19:26:23 +02:00
plugins Computation and checking of module hashes 2024-05-28 11:43:50 +02:00
scalc Remove extra space 2024-06-03 09:06:57 +02:00
scopelang Add multiple typing errors using delayed errors 2024-06-19 14:52:31 +02:00
shared_ast Fix typing error message consistency 2024-06-19 14:52:31 +02:00
surface Generate tests reports from 'clerk test' 2024-06-19 16:10:26 +02:00
verification More precise positions for operators throughout 2024-04-30 16:35:08 +02:00
catala_web_interpreter.ml Rewriting message calls to use the new intf 2024-04-10 19:26:23 +02:00
catala.ml replace let _ by let () or add type annotation 2023-09-09 22:02:39 +02:00
driver.ml Small code cleanups 2024-06-19 16:10:15 +02:00
driver.mli Move global options of Cli to their own module 2024-03-19 15:18:35 +01:00
dune Update opam file according to opam-repository reviews 2024-05-06 21:39:58 +02:00
index.mld More doc 2023-06-18 16:08:16 +02:00
plugin.ml Rewriting message calls to use the new intf 2024-04-10 19:26:23 +02:00
plugin.mli Move global options of Cli to their own module 2024-03-19 15:18:35 +01:00
tests.ml Generalise the definition of lists of nested binders 2024-02-09 18:33:41 +01:00