this is more consistent, avoids empty stamp files and should make things simpler
overall.
The slightly tricky `-C` option is added to Catala so that it can be run from
_build while paths to source and destination files are still specified relative
to CWD (Ninja doesn't provide string manipulation, so otherwise we would have to
explicit both the `_build/dir/` and `dir/` versions of each path).
* Obsolete code for included tests has been removed
* The engine uses a proper lexer and is much simplified
* An inline test in the middle of the file now only "sees" the file up to that
point. This fixes an issue where we had spurious errors when a type error was
added at the end of a file, and it would pop up in tests before it. This makes
files including many tests much more practical.
* diffing and resetting the tests has been reintroduced (done at the moment in
Ninja, but for more control (count number of failed tests, etc.) we could put it
back into Clerk at some point
* The Catala CLI can now take an input from stdin (with the possibility to link
a (possibly fake) on-disk file for error reporting and file locations ; this
is useful for running tests)
We need a concrete intermediate target for e.g. transitive uses of `> Include`
for Ninja to correctly handle them.
Of course we could also unroll all transitive dependencies, but meh.
Note also that now tests now just generate the outputs but facilities for
diffing and resetting are temporarily absent.
Rather than require all files to be listed on the command-line (and having to
check consistency with `> Using` directives), the main catala CLI is now a bit
more clever.
⇒ There is a new assumption that a module name definition must match the file
name (up to case and extension) — with appropriate error handling to enforce it.
In exchange, `> Using` directives are now used to more transparently lookup the
appropriate `.catala_*` interfaces and the compiled artifacts for the used modules (handling transitive dependencies), with just standard `-I` flags for when they need to be looked up in different places.
A recent patch introduced custom terms in expressions manipulated by the
interpreter. For typing reasons, a traversal is done to extend the supplied
expression with these custom terms — it's functionally the identity, but, due to
the fact that the type-checker can't infer covariance of our AST terms on their
phantom parameter, playing by the rules imposes a full traversal + rebuild.
Without resorting to a (constrained) `Obj.magic`, this patch avoids extra
intermediate conversions, which is enough to cut out the huge extra cost we were
incurring.
Closes#516
rather than scattered in structures
The context is still hierarchical for defs though, so one needs to retrieve the
path to lookup in the correct context for info. Exceptions are enums and struct
defs, which are re-exposed at toplevel.
This makes sure `catala module` finds the local runtime when run from the catala
source tree; and fixes lookup of the catala exec on custom uses of `clerk runtest`.
... and add a custom printer
Since this is a very common bug, this patch should gain us a lot of time when
debugging uncaught Not_found errors, because the element not found can now be
printed straight away without the need for further debugging.
The small cost is that one should remember to catch the correct specialised
`Foo.Map.Not_found _` exception rather than the standard `Not_found` (which
would type-check but not catch the exception). Using `find_opt` should be
preferred anyway.
Note that the other functions from the module `Map` that raise `Not_found` are
not affected ; these functions are `choose`, `min/max_binding`,
`find_first/last` which either take a predicate or fail on the empty map, so it
wouldn't make sense for them (and we probably don't use them much).
the small downside is that we lose a little bit of granularity by making
defining all the types at once (with `type ... and`) ; but one well-placed
annotation seems to be enough.
also discarded the `iter` visitor that was unused.
- We're able to say from the parser what the user could have written. It may not be complete due to the acceptable function of Menhir : it is only an indication given to the user (and not intended to be an adaptive documentation)
- .mli file added : module interface for suggestions
- Add a test that provides a typographical or a logical error.
- Documentation and type / name changes for suggestions
- Clearer (more detailed) error messages (see parser.messages, parser_driver.ml ->)
--> Parser_driver.ml : we're more precise and show all possible instructions to the user.
- Suggestions.ml : unimportant display changes + renaming
- suggestions.ml : improve details + algorithm implementation (see @Altgr reviews) + comments
- suggestions implementation : since we want to keep the same structure to display suggestion messages, Suggestion is no longer a formatted string but a string list. The goal is to have a similar formatted string for each suggestion, no matter what the error message is.
--> This involves changes to suggestions.mli, compiler/desugared/from_surface.ml (most important one) and compiler/surface/parser_driver.ml
This patch functorises the generic expression printer, in order to be able to
re-use it for end-user printing.
It makes it possible to have an end-user, localised printer that shares the code
for e.g. priority and automatic parens handling.
A generic AST rewriting that disambiguates variables (very simple to write with
bindlib) is also added and used in the OCaml backend for something safer than
just appending `_user` (-- this also handles clashing variables that could be
introduced during compilation which would have generated wrong code before this)
Finally, the `explain` plugin is adapted to use the new printer.
Ah, and `String.format_t` was tweaked to correctly print strings that might
contain unicode without breaking alignment, and should be used instead of
`format_string` or `%s` whenever unicode can be expected.
- Use separate functions for successive passes in module `Driver.Passes`
- Use other functions for end results printing in module `Driver.Commands`
As a consequence, it is much more flexible to use by plugins or libs and we no
longer need the complex polymorphic variant parameter.
This patch leverages previous changes to use Cmdliner subcommands and
effectively specialises the flags of each Catala subcommand.
Other changes include:
- an attempt to normalise the generic options and reduce the number of global
references. Some are ok, like `debug` ; some would better be further cleaned up,
e.g. the ones used by Proof backend were moved to a `Proof.globals` module and
need discussion. The printer no longer relies on the global languages and prints
money amounts in an agnostic way.
- the plugin directory is automatically guessed and loaded even in dev setups.
Plugins are shown by the main `catala` command and listed in `catala --help`
- exception catching at the toplevel has been refactored a bit as well; return
codes are normalised to follow the manpage and avoid codes >= 128 that are
generally reserved for shells.
Update tests
The upside of this is that each command can define specific flags ; there is a
small loss of backwards-compatibility in that the command needs to be the first
argument.
`catala --help` will now only show a summary of commands, with more specific
manpages shown on `catala CMD --help`.
Another point is that the plugin interface is extended to allow plugins to be
registered as subcommands and have their own flags (this will be very useful for
adding flags to the lazy/dot/explanation plugin that has many options).
Note that no efforts has yet been made to specialise the options, the previous
type was just made global for all subcommands.
(first working dynload test with compilation done by manual calls to ocaml)
A few pieces of the puzzle:
* Loading of interfaces only from Catala files
* Registration of toplevel values in modules compiled to OCaml, to allow access
using dynlink
* Shady conversion from OCaml runtime values to/from Catala expressions, to
allow interop (ffi) of compiled modules and the interpreter
Two interdependent changes here:
1. Enforce all instances of Shared_ast.gexpr to use the generic type for marks.
This makes the interfaces a tad simpler to manipulate: you now write
`('a, 'm) gexpr` rather than `('a, 'm mark) gexpr`.
2. Define a polymorphic `Custom` mark case for use by pass-specific annotations.
And leverage this in the typing module
The module is renamed to `Mark`, and functions renamed to avoid redundancy:
`Marked.mark` is now `Mark.add`
`Marked.unmark` is now `Mark.remove`
`Marked.map_under_mark` is now simply `Mark.map`
etc.
`Marked.same_mark_as` is replaced by `Mark.copy`, but with the arguments
swapped (which seemed more convenient throughout)
Since a type `Mark.t` would indicate a mark, and to avoid confusion, the type
`Marked.t` is renamed to `Mark.ed` as a shorthand for `Mark.marked` ; this part
can easily be removed if that's too much quirkiness.
A module without mli is ok as long as it only contains types
Here we already stretch it a bit with some functor applications, but having
toplevel values defeats the expectation that you can safely `open` this module.
- Fix the printer for scopes
- Improve the printer for struct types
- Remove `Print.expr'`. Use `Expr.format` as the function with simplified arguments instead.
- `Print.expr` no longer needs the context
- This removes the need for `expr ~debug` + `expr_debug` ;
use `Print.expr` for normal (non-debug) output,
and `Print.expr' ?debug ()` for possibly debug output.
- This improves consistency of debug expr output in many places
- Prints simplified operators (without type suffix) in non-verbose mode
(this patch also fixes some cases of `Expr.skip_wrappers` and leverages the
binder equality provided by Bindlib)
To try it (without installing Catala):
```shell-session
$ make plugins
$ export CATALA_PLUGINS=_build/default/compiler/plugins
$ dune exec -- catala lazy examples/aides_logement/tests/tests_calcul_apl_locatif.catala_fr -s Exemple2
```
Keep in mind that this is a work-in-progress prototype :)
I made some changes in the meantime, and had to factorise e.g. the handling of
the `EEmptyError` case, but this is the simple approach type-wise of making the
function type for `∀ 'a. 'a —> 'a` (with `assert false` match cases), then
restricting its type do `dcalc` or `lcalc` in the `.mli`.
The phantom polymorphic variant qualifying AST nodes is reversed:
- previously, we were explicitely restricting each AST node to the passes where it belonged using a closed type (e.g. `[< dcalc | lcalc]`)
- now, each node instead declares the "feature" it provides using an open type (e.g. `[> 'Exceptions ]`)
- then the AST for a specific pass limits the features it allows with a closed type
The result is that you can mix and match all features if you wish,
even if the result is not a valid AST for any given pass. More
interestingly, it's now easier to write a function that works on
different ASTs at once (it's the inferred default if you don't write a
type restriction).
The opportunity was also taken to simplify the encoding of the
operators, which don't need a second type parameter anymore.
