First I was disappointed that we couldn't convert closure environment properly
because of their opaque nature (native/interpreted conversion is based on the
Catala types) ; but after more thought it's actually unnecessary to convert them
at all since we are guaranteed that they can't be consumed outside of their
realm.
it's much simpler to handle down the line if they have a uniform structure;
empty tuples are easily converted into unit types when translating to OCaml.
This avoids differences in test results depending on wether closure conversion
is enabled or not: the functional values within structure are a different type
internally but with this patch they are printed the same.
Support for manipulating toplevel functions as values was buggy, because the
recursion after eta-expansion would fall into the pattern for a `let..in` and
not do the expected transformation.
The patch explicitely builds the closure in that case, avoiding such issues with
recursion.
This is a proper replacement for the previous shell-based placeholder hack.
Here is a summary:
- `clerk runtest` (normally run by ninja) is much extended:
* besides generating the test@out file, it checks individual tests for success
and can write a report file containing their status, and the positions for
their (expected/current) outputs (this uses `Marshal`)
* it now handles out-tests directly in addition to inline-tests, for which
it generates the separate output file ; they are included in the report
- ninja is now tasked with building all the test reports (which shouldn't fail);
for directories, individual reports are concatenated (as before).
Removing intermediate report rules, and out-test rules means that the ninja
file is much simplified.
- then, clerk takes back control, reads the final reports and formats them in a
user-friendly way. Printing the reports may imply running `diff` internally.
In particular, the commands to easily reproduce each test are provided.
Resetting the test results if required is also done directly by clerk, at this
stage.
A few switches are available to customise the output, but I am waiting for some
feedback before deciding what to make available from the CLI.
The `clerk report` command is available to manually explore test reports, but
normally the processing is done directly at the end of `clerk test` (i.e. ninja
will no longer call that command)
some of the types (in particular, in hoisted closures) could not be
reconstructed afterwards. This properly propagates the types, including to
closure deconstruction time, giving additional insurance; and allowing
monomorphisation not to choke on the result.
NOTE: This is a temporary solution
A future approach could be to have Catala generate a module loader (with the
proper hash), relieving the user implementation from having to do the
registration.
This includes a few separate changes:
- pass visibility information of declarations (depending on wether the
declaration was in a ```catala-metadata block or not)
- add reasonable hash computation functions to discriminate the interfaces. In
particular:
* Uids have a `hash` function that depends on their string, but not on their
actual uid (which is not stable between runs of the compiler) ; the existing
`hash` function and its uses have been renamed to `id`.
* The `Hash` module provides the tools to properly combine hashes, etc. While
we rely on `Hashtbl.hash` for the atoms, we take care not to use it on any
recursive structure (it relies on a bounded traversal).
- insert the hashes in the artefacts, and properly check and report those (for
OCaml)
**Remains to do**:
- Record and check the hashes in the other backends
- Provide a way to get stable inline-test outputs in the presence of module
hashes
- Provide a way to write external modules that don't break at every Catala
update.
This is a first step into unifying trace handling. This patch only affects the
interpreter, by delegating trace recording to the already existing runtime
functions.
At end of interpretation, it recovers the registered trace from the runtime, and
prints it.
NOTE: there are some limitations due to this approach, as runtime values going
through this interface have to be converted to the "runtime embedded" type. In
particular, functions can no longer be printed (which makes full sense if we
want it to happen in the same way in compiled code) ; some information, like
types, is lost, but it didn't appear to be used.
Also, a specific printer had to be added for runtime values (but it's very
simple so that shouldn't be a problem).
@denismerigoux I'd like your input on how well this goes for your use-cases.
Further work should probably be cleanup and unification of the runtime logging
interfaces ; there is already code for re-structuring the traces, printing to
JSON, etc. which could be common to runtime and interpreter.
mostly reverting to the ones the interpreter was printing ; for the case of
divisions, we choose to point to the denominator instead of the operator as it's
where the only possible error (division by zero) comes from.
Positions within the Default terms are specially important since they can come
from separate definitions in the source (before this, we would be falling back
to the single declaration).
