Previously we had some heuristics in the backends trying to achieve this with a
lot of holes ; this should be much more solid, relying on `Bindlib` to do the
correct renamings.
**Note1**: it's not plugged into the backends other than OCaml at the moment.
**Note2**: the related, obsolete heuristics haven't been cleaned out yet
**Note3**: we conservatively suppose a single namespace at the moment. This is
required for e.g. Python, but it forces vars named like struct fields to be
renamed, which is more verbose in e.g. OCaml. The renaming engine could be
improved to support different namespaces, with a way to select how to route the
different kinds of identifiers into them.
Similarly, customisation for what needs to be uppercase or lowercase is not
available yet.
**Note4**: besides excluding keywords, we should also be careful to exclude (or
namespace):
- the idents used in the runtime (e.g. `o_add_int_int`)
- the dynamically generated idents (e.g. `embed_*`)
**Note5**: module names themselves aren't handled yet. The reason is that they
must be discoverable by the user, and even need to match the filenames, etc. In
other words, imagine that `Mod` is a keyword in the target language. You can't
rename a module called `Mod` to `Mod1` without knowing the whole module context,
because that would destroy the mapping for a module already called `Mod1`.
A reliable solution would be to translate all module names to e.g.
`CatalaModule_*`, which we can assume will never conflict with any built-in, and
forbid idents starting with that prefix. We may also want to restrict their
names to ASCII ? Currently we use a projection, but what if I have two modules
called `Là` and `La` ?
*Disclaimer*: This is intended for discussion
My impression is that the with-exceptions backend is to be superseded by the
without-exception backend, which is more general and more efficient. Therefore,
seeing the added complexity of maintaining the two in parallel, I see no good
reason to keep the with-exceptions version now that the equivalence of their
semantics have been proved.
It will also be nice to reduce divergences between the different backends ; and
this should make further simplifications possible (e.g. some thunkings may no
longer be needed)
Of course I am ready to hear arguments in favor of keeping it, be it in the mid-
or long-term.
This patch removes the `--avoid-exceptions` flag, making it the only option, and
the corresponding `with_exceptions` variant of the dcalc->lcalc translation. It
doesn't do further simplifications.
Not much there at the moment, but being able to specify the include directories
is already pretty useful to run clerk directly e.g. on `catala-examples`.
(you had to explicitely specify variable `CATALA_INCLUDE`, the `-I` flags or to
go through `make` without that)
This relies less on specific color flags of GNU diff, and reformats and
colorises the output.
(it may still depend on the specific layout of GNU diff with the `-y` flag
though)
This is a proper replacement for the previous shell-based placeholder hack.
Here is a summary:
- `clerk runtest` (normally run by ninja) is much extended:
* besides generating the test@out file, it checks individual tests for success
and can write a report file containing their status, and the positions for
their (expected/current) outputs (this uses `Marshal`)
* it now handles out-tests directly in addition to inline-tests, for which
it generates the separate output file ; they are included in the report
- ninja is now tasked with building all the test reports (which shouldn't fail);
for directories, individual reports are concatenated (as before).
Removing intermediate report rules, and out-test rules means that the ninja
file is much simplified.
- then, clerk takes back control, reads the final reports and formats them in a
user-friendly way. Printing the reports may imply running `diff` internally.
In particular, the commands to easily reproduce each test are provided.
Resetting the test results if required is also done directly by clerk, at this
stage.
A few switches are available to customise the output, but I am waiting for some
feedback before deciding what to make available from the CLI.
The `clerk report` command is available to manually explore test reports, but
normally the processing is done directly at the end of `clerk test` (i.e. ninja
will no longer call that command)
NOTE: This is a temporary solution
A future approach could be to have Catala generate a module loader (with the
proper hash), relieving the user implementation from having to do the
registration.
This includes a few separate changes:
- pass visibility information of declarations (depending on wether the
declaration was in a ```catala-metadata block or not)
- add reasonable hash computation functions to discriminate the interfaces. In
particular:
* Uids have a `hash` function that depends on their string, but not on their
actual uid (which is not stable between runs of the compiler) ; the existing
`hash` function and its uses have been renamed to `id`.
* The `Hash` module provides the tools to properly combine hashes, etc. While
we rely on `Hashtbl.hash` for the atoms, we take care not to use it on any
recursive structure (it relies on a bounded traversal).
- insert the hashes in the artefacts, and properly check and report those (for
OCaml)
**Remains to do**:
- Record and check the hashes in the other backends
- Provide a way to get stable inline-test outputs in the presence of module
hashes
- Provide a way to write external modules that don't break at every Catala
update.
It makes it easier to batch-set them when testing a given repo with different
sets of options. Direct flags should still be preferred in general, of course.
Ensuring messages don't print overlong lines still requires some manual work:
- if they don't contain any `Format` directives (`%` or `@`), use `"%a"
Format.pp_print_text` to turn word-wrapping on.
- otherwise replace spaces with `@ ` to mark possible cutting points, as soon
that it's possible the line will get over 80 chars (most often, this means
starting before the first `%a`)
- This adds a `catala depends` command that recursively tracks module dependency.
It can then be used by Clerk for linking.
- Generation of cmo object files are added for OCaml (we only built native
objects, but jsoo requires bytecode).
- Some fixes to the generation of value embed/deembed shims (related to types
coming from different modules ; add support for options ; etc.)
