Previously we had some heuristics in the backends trying to achieve this with a
lot of holes ; this should be much more solid, relying on `Bindlib` to do the
correct renamings.
**Note1**: it's not plugged into the backends other than OCaml at the moment.
**Note2**: the related, obsolete heuristics haven't been cleaned out yet
**Note3**: we conservatively suppose a single namespace at the moment. This is
required for e.g. Python, but it forces vars named like struct fields to be
renamed, which is more verbose in e.g. OCaml. The renaming engine could be
improved to support different namespaces, with a way to select how to route the
different kinds of identifiers into them.
Similarly, customisation for what needs to be uppercase or lowercase is not
available yet.
**Note4**: besides excluding keywords, we should also be careful to exclude (or
namespace):
- the idents used in the runtime (e.g. `o_add_int_int`)
- the dynamically generated idents (e.g. `embed_*`)
**Note5**: module names themselves aren't handled yet. The reason is that they
must be discoverable by the user, and even need to match the filenames, etc. In
other words, imagine that `Mod` is a keyword in the target language. You can't
rename a module called `Mod` to `Mod1` without knowing the whole module context,
because that would destroy the mapping for a module already called `Mod1`.
A reliable solution would be to translate all module names to e.g.
`CatalaModule_*`, which we can assume will never conflict with any built-in, and
forbid idents starting with that prefix. We may also want to restrict their
names to ASCII ? Currently we use a projection, but what if I have two modules
called `Là` and `La` ?
This case is really acrobatic, because we are interfacing code compiled to
OCaml (from lcalc) with code from dcalc, and the two have a different
representation for default terms. It... seems to work though.
A more reasonable solution, if there are problems with this or it reveals too
fragile, would be to enforce interpreting at the lcalc level as soon as you want
to link compiled modules.
HandleExceptions only takes an array of exceptions, and returns Some if only one
of them is Some, None if they are all None, or raises a conflict error
otherwise.
The compilation of default terms then wraps this in a match (for the result of
HandleExceptions), and an if-then-else (for the justification-consequence in the
None case).
This avoids the complexity of having to handle thunked functions as arguments.
First I was disappointed that we couldn't convert closure environment properly
because of their opaque nature (native/interpreted conversion is based on the
Catala types) ; but after more thought it's actually unnecessary to convert them
at all since we are guaranteed that they can't be consumed outside of their
realm.
This avoids differences in test results depending on wether closure conversion
is enabled or not: the functional values within structure are a different type
internally but with this patch they are printed the same.
Support for manipulating toplevel functions as values was buggy, because the
recursion after eta-expansion would fall into the pattern for a `let..in` and
not do the expected transformation.
The patch explicitely builds the closure in that case, avoiding such issues with
recursion.
some of the types (in particular, in hoisted closures) could not be
reconstructed afterwards. This properly propagates the types, including to
closure deconstruction time, giving additional insurance; and allowing
monomorphisation not to choke on the result.
NOTE: This is a temporary solution
A future approach could be to have Catala generate a module loader (with the
proper hash), relieving the user implementation from having to do the
registration.
This includes a few separate changes:
- pass visibility information of declarations (depending on wether the
declaration was in a ```catala-metadata block or not)
- add reasonable hash computation functions to discriminate the interfaces. In
particular:
* Uids have a `hash` function that depends on their string, but not on their
actual uid (which is not stable between runs of the compiler) ; the existing
`hash` function and its uses have been renamed to `id`.
* The `Hash` module provides the tools to properly combine hashes, etc. While
we rely on `Hashtbl.hash` for the atoms, we take care not to use it on any
recursive structure (it relies on a bounded traversal).
- insert the hashes in the artefacts, and properly check and report those (for
OCaml)
**Remains to do**:
- Record and check the hashes in the other backends
- Provide a way to get stable inline-test outputs in the presence of module
hashes
- Provide a way to write external modules that don't break at every Catala
update.
This is a first step into unifying trace handling. This patch only affects the
interpreter, by delegating trace recording to the already existing runtime
functions.
At end of interpretation, it recovers the registered trace from the runtime, and
prints it.
NOTE: there are some limitations due to this approach, as runtime values going
through this interface have to be converted to the "runtime embedded" type. In
particular, functions can no longer be printed (which makes full sense if we
want it to happen in the same way in compiled code) ; some information, like
types, is lost, but it didn't appear to be used.
Also, a specific printer had to be added for runtime values (but it's very
simple so that shouldn't be a problem).
@denismerigoux I'd like your input on how well this goes for your use-cases.
Further work should probably be cleanup and unification of the runtime logging
interfaces ; there is already code for re-structuring the traces, printing to
JSON, etc. which could be common to runtime and interpreter.
mostly reverting to the ones the interpreter was printing ; for the case of
divisions, we choose to point to the denominator instead of the operator as it's
where the only possible error (division by zero) comes from.
- Clearly distinguish Exceptions from Errors. The only catchable exception
available in our AST is `EmptyError`, so the corresponding nodes are made less
generic, and a node `FatalError` is added
- Runtime errors are defined as a specific type in the OCaml runtime, with a
carrier exception and printing functions. These are used throughout, and
consistently by the interpreter. They always carry a position, that can be
converted to be printed with the fancy compiler location printer, or in a
simpler way from the backends.
- All operators that might be subject to an error take a position as argument,
in order to print an informative message without relying on backtraces from
the backend
Module names must be capitalised (start with a capital letter), and the name of
the file on disk must match ; however, matching up to capitalisation is allowed,
i.e. the file on disk can start with a lowercase letter.
A mismatch between Clerk assuming generated module artifacts would match the
capitalised module name, and `catala depends` matching the names of files on
disk (because it would otherwise mean treating dependencies differently
depending on if they originate from modules or not) was causing "file not found"
errors later on in the compilation chain.
This patch enforces that the capitalisation of the original file name on
disk (which is always known) takes precedence in Clerk, matching the behaviour
of `catala depends` and fixing the issue. It's also actually a small
simplification in Clerk code.
This was a pending TODO: now the Catala program compiled into OCaml should
return better messages and a little more information about uncaught exceptions.
Note that this also concerns, at the moment, compiled modules called from the
Catala interpreter: in this case, it's already better than nothing, but what we
need is proper interoperation between the runtime exceptions and the interpreter
handling (`EmptyError` should already be handled properly since it is critical
to the computation flow, but "error" exceptions are left uncaught and will kill
the interpreter).
This may be part of a bigger task on unifying the output of the runtime and
toplevel, which also concerns computation traces.
Note 2: All runtime exceptions don't have a position available, which is quite
unfortunate when your program hits an error. With `OCAMLRUNPARAM=b` and if
compiled with `-g` (which should normally be the case), you can get an OCaml
backtrace but that's not very friendly. Ideas for improvement:
- The runtime could force-enable backtrace recording (`Printexc.record_backtrace
true`) to supersede the need for `OCAMLRUNPARAM`. We can also record our own
handler to print the file position and/or backtrace in the way we see fit
- The printer of OCaml code in Catala could insert line directives so that the
positions in the backtrace actually trace automatically back to the Catala
code
- If we don't want to leverage any OCaml machinery in this way, the compiler
should add position information to any operator that might fail (e.g.
divisions, date comparisons, etc.).
Note that running in trace mode might already help pinpoint the location of the
error ?
As discussed in #549
NOTE: This implements only the direct tuple member access (syntax `foo.N` with N a
number)
- It seems more efficient to wait for the general pattern-matching rewrite to
handle pattern-matching on tuples
- Until then we keep the (now obsolete) `let (x, y) = pair in x` syntax, to
leave time for updates, but we won't be documenting it
Closes#592
A new node is added in `desugared`, and translated into an exploded structure
literal during translation to `scopelang`. The main reason to put it there is
that it needs to be after disambiguation, since that is used to discover the
type of the structure that is being updated.
Ensuring messages don't print overlong lines still requires some manual work:
- if they don't contain any `Format` directives (`%` or `@`), use `"%a"
Format.pp_print_text` to turn word-wrapping on.
- otherwise replace spaces with `@ ` to mark possible cutting points, as soon
that it's possible the line will get over 80 chars (most often, this means
starting before the first `%a`)
Lots of tests have a new warning because they were calling subscopes without
using their outputs. A better solution could be to mark these subscopes as
`output`, now that it's possible !
