This PR adds the `match-to-case` Core transformation. This transforms
pattern matching nodes to a sequence of case and let nodes.
## High level description
Each branch of the match is compiled to a lambda. In the combined match
Each branch of the match is compiled to a lambda. These lambdas are
combined in nested lets and each lambda is called in turn as each branch
gets checked. The lambda corresponding to the first branch gets called
first, if the pattern match in the branch fails, the lambda
corresponding to the next branch is called and so on. If no branches
match then a lambda is called which returns a fail node.
Conceptually:
<table>
<tr>
<td>
Core
</td>
<td>
Transformed
</td>
</tr>
<tr>
<td>
```
match v1 .. vn {
b1
b2
...
bk
}
```
</td>
<td>
```
λ
let c0 := λ FAIL in
let ck := λ {...} in
...
let c1 := λ {...} in
c1 v1 ... vn
```
</td>
</tr>
</table>
The patterns on each branch are compiled to either let bindings (pattern
binders) or case expressions (constructor patterns).
Auxillary bindings are added in the case of nested constructor patterns.
The default branch in each case expression has a call to the lambda
corresponding to the next branch of the match. This is because the
default
branch is reached if the pattern match fails.
<table>
<tr>
<td>
Pattern match
</td>
<td>
Transformed
</td>
</tr>
<tr>
<td>
```
suc (suc n) ↦ n
```
</td>
<td>
```
case ?$0 of {
suc arg_8 := case ?$0 of {
suc n := let n := ?$0 in n$0;
_ := ?$2 ?$1
};
_ := ?$1 ?$0
}
```
</td>
</tr>
</table>
The body of each branch is wrapped in let bindings so that the indicies
of bound
variables in the body point to the correct variables in the compiled
expression.
This is necessary because the auxiliary bindings added for nested
constructor
patterns will cause the original indicies to be offset.
Finally, the free variables in the match branch body need to be shifted
by all the bindings we've added as part of the compilation.
## Examples
### Single wildcard
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat;
f _ := 1;
```
</td>
<td>
```
λ? match ?$0 with {
_ω309 ↦ ? 1
}
```
</td>
<td>
```
λ? let ? := λ? fail "Non-exhaustive patterns" in
let ? := λ? let _ω309 := ?$0 in
let _ω309 := ?$0 in 1 in
?$0 ?$2
```
</td>
</tr>
</table>
### Single binder
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat;
f n := n;
```
</td>
<td>
```
λ? match ?$0 with {
n ↦ n$0
}
```
</td>
<td>
```
λ? let ? := λ? fail "Non-exhaustive patterns" in
let ? := λ? let n := ?$0 in
let n := ?$0 in n$0 in
?$0 ?$2
```
</td>
</tr>
</table>
### Single Constructor
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat;
f (suc n) := n;
```
</td>
<td>
```
λ? match ?$0 with {
suc n ↦ n$0
}
```
</td>
<td>
```
λ? let ? := λ? fail "Non-exhaustive patterns" in let ? := λ? case ?$0 of {
suc n := let n := ?$0 in let n := ?$0 in n$0;
_ := ?$1 ?$0
} in ?$0 ?$2
```
</td>
</tr>
</table>
### Nested Constructor
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat;
f (suc (suc n)) := n;
```
</td>
<td>
```
λ? match ?$0 with {
suc (suc n) ↦ n$0
}
```
</td>
<td>
```
λ? let ? := λ? fail "Non-exhaustive patterns" in let ? := λ? case ?$0 of {
suc arg_8 := case ?$0 of {
suc n := let n := ?$0 in let n := ?$0 in n$0;
_ := ?$2 ?$1
};
_ := ?$1 ?$0
} in ?$0 ?$2
```
</td>
</tr>
</table>
### Multiple Branches
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat;
f (suc n) := n;
f zero := 0;
```
</td>
<td>
```
λ? match ?$0 with {
suc n ↦ n$0;
zero ↦ ? 0
}
```
</td>
<td>
```
λ? let ? := λ? fail "Non-exhaustive patterns" in let ? := λ? case ?$0 of {
zero := ? 0;
_ := ?$1 ?$0
} in let ? := λ? case ?$0 of {
suc n := let n := ?$0 in let n := ?$0 in n$0;
_ := ?$1 ?$0
} in ?$0 ?$3
```
</td>
</tr>
</table>
### Nested case with captured variable
<table>
<tr>
<td> Juvix </td> <td> Core </td> <td> Transformed Core </td>
</tr>
<tr>
<td>
```
f : Nat -> Nat -> Nat;
f n m := case m
| suc k := n + k;
```
</td>
<td>
```
f = λ? λ? match ?$1, ?$0 with {
n, m ↦ match m$0 with {
suc k ↦ + n$2 k$0
}
}
```
</td>
<td>
```
λ? λ?
let ? := λ? λ? fail "Non-exhaustive patterns" in
let ? := λ? λ? let n := ?$1 in let m := ?$1 in let n := ?$1 in let m := ?$1 in
let ? := λ? fail "Non-exhaustive patterns" in let ? := λ? case ?$0 of {
suc k := let k := ?$0 in let k := ?$0 in + n$6 k$0;
_ := ?$1 ?$0
} in ?$0 m$2 in ?$0 ?$3 ?$2
```
</td>
</tr>
</table>
## Testing
The `tests/Compilation/positive` tests are run up to the Core evaluator
with `match-to-case` and `nat-to-int` transformations on Core turned on.
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
- Closes#1793.
Now, if the body of a function clause does not fit in a line, the body
will start indented in the next line.
The example presented in the linked issue is now formatted thus:
```
go n s :=
if
(s < n)
(go (sub n 1) s)
(go n (sub s n) + go (sub n 1) s);
```
Core transformations apply to the whole InfoTable, the REPL needs to
apply Core transformations to the single node that it compiles from the
user input string.
The solution in this commit is to:
1. Compile the input string as before to obtain a Core Node.
2. Add this Node to a copy of the Core InfoTable for the loaded file.
3. Apply the (CLI specified) Core transformations to this InfoTable.
4. Extract the (now transformed) Node from the InfoTable.
We can think of a way to improve this, maybe when we tackle allowing the
user to make new definitions in the REPL.
As soon as compilation of pattern matching is complete we should enable
some (all?) Core transformations by default.
Example:
At the moment we get the following result in the REPL:
```
juvix repl
...
Stdlib.Prelude> 1 + 1
suc (suc zero)
```
After this commit we can turn on `nat-to-int` transformation:
```
juvix repl -t nat-to-int
Stdlib.Prelude> 1 + 1
2
```
* Part of https://github.com/anoma/juvix/issues/1531
This PR:
- Closes#1647
It gives compilation errors for language features that require more
substantial support (recursion, polymorphism). The additional features
are to be implemented in future separate PRs.
* Adds a new target `geb` to the CLI command `juvix dev core compile`,
which produces a `*.geb` output file in the `.juvix-build` directory.
* Adds a few tests. These are not yet checked automatically because
there is no GEB evaluator; checking the `*.geb` output would be too
brittle.
Before this change, nested as-patterns (i.e as-patterns binding
arguments to constructors) were not translated to Core pattern binders.
This meant that the following function would crash the compiler:
```
f : List Nat -> List Nat;
f (x :: a@(x' :: xs)) := a;
f _ := nil;
```
i.e the nested as-pattern `a` was ignored in the internal to core
translation.
This commit translates each as-pattern to a Core `PatternBinder`.
* Fixes https://github.com/anoma/juvix/issues/1788
* Fixes https://github.com/anoma/juvix/issues/1738
- Closes#1637.
A function type signature is now allowed to have a body. This is valid
for both top level and let definitions.
```
not : Bool -> Bool := λ {
| true := false
| false := true
};
```
The integer to Nat translation in the Internal to Core translation
depends on both Nat and Bool builtin types being in the InfoTable.
544bddba43/src/Juvix/Compiler/Core/Translation/FromInternal.hs (L67)
If the root module does not contain an explicit reference to the builtin
Bool (for example) then builtin Bool type is filtered out by the
reachability analysis and therefore is not available at transltaion
time.
In this commit we add both builtin Nat and builtin Bool as start nodes
in the reachability analysis to guarantee that they will not be filtered
out.
- Fixes https://github.com/anoma/juvix/issues/1774
- Fixes#1723
- It refactors parsing/scoping so that the scoper does not need to read
files or parse any module. Instead, the parser takes care of parsing all
the imported modules transitively.
This PR adds some maintenance at different levels to the CI config, the
Make file, and formatting.
- Most of the actions used by the CI related to haskell, ormolu, hlint
and pre-commit have been updated because Github requires NodeJS 16. This
change removes all the old warnings related to nodeJs.
In the case of ormolu, the new version makes us format some files that
were not formatted before, similarly with hlint.
- The CI has been updated to use the latest version of the Smoke testing
framework, which introduced installation of the dependencies for Linux
(libicu66) and macOS (icu4c) in the CI. In the case of macOS, the CI
uses a binary for smoke. For Linux, we use stack to build smoke from the
source. The source here is in a fork of [the official Smoke
repo](https://github.com/SamirTalwar/smoke). Such includes some
features/changes that are not yet in the official repo.
- The Makefile runs the ormolu and hlint targets using as a path for the
binaries the environment variables ORMOLU and HLINT. Thus, export those
variables in your environment before running `make check,` `make format`
or `make hlint`. Otherwise, the Makefile will use the binaries provided
by `stack`.
Co-authored-by: Paul Cadman <git@paulcadman.dev>
This PR redefines the `html` command unifying our previous subcommands
for the HTML backend. You should use the command in the following way to
obtain the same results as before:
- `juvix html src.juvix` -> `juvix html src.juvix --only-source`
- `juvix dev doc src.juvix` -> `juvix html src.juvix`
- Other fixes here include the flag `--non-recursive`, which replaces
the previous behavior in that we now generate all the HTML recursively
by default.
- The flag `--no-print-metadata` is now called `--no-footer`
- Also, another change introduced by this PR is asset handling; for
example, with our canonical Juvix program,
the new output is organized as follows.
```
juvix html HelloWorld.juvix --only-source && tree html/
Copying assets files to test/html/assets
Writing HelloWorld.html
html/
├── assets
│ ├── css
│ │ ├── linuwial.css
│ │ ├── source-ayu-light.css
│ │ └── source-nord.css
│ ├── images
│ │ ├── tara-magicien.png
│ │ ├── tara-seating.svg
│ │ ├── tara-smiling.png
│ │ ├── tara-smiling.svg
│ │ ├── tara-teaching.png
│ │ └── tara-teaching.svg
│ └── js
│ ├── highlight.js
│ └── tex-chtml.js
└── HelloWorld.html
├── Stdlib.Data.Bool.html
├── Stdlib.Data.List.html
├── Stdlib.Data.Maybe.html
├── Stdlib.Data.Nat.html
├── Stdlib.Data.Ord.html
├── Stdlib.Data.Product.html
├── Stdlib.Data.String.html
├── Stdlib.Function.html
├── Stdlib.Prelude.html
└── Stdlib.System.IO.html
```
In addition, for the vscode-plugin, this PR adds two flags,
`--prefix-assets` and `--prefix-url`, for which one provides input to
help vscode find resource locations and Juvix files.
PS. Make sure to run `make clean` the first time you run `make install`
for the first time.
The internal to core translation was removing implicit arguments from
function definitions and applications. This is incorrect as the implicit
bindings are required when translating the following (in `csuc`, the
binding of the implicit argument is required in an application on the
rhs):
```
Num : Type;
Num := {A : Type} → (A → A) → A → A;
csuc : Num → Num;
csuc n {_} f := f ∘ n {_} f;
```
Apart from removing this filter from function and application
translation, this required the following changes:
ConstructorInfo:
The _constructorArgsNum field must include the number of type parameters
of its inductive type.
PatternConstructorApp:
The pattern arguments must include wildcards for the implicit type
parameters passed to the constructor.
BuiltinIf:
The BuiltinIf expression is passed an implicit type argument that must
be removed when translating to Core if.
LitString:
A literal string is a function with an implcit type argument. So this
must be a translated to a lambda where the type argument is ignored.
Fixes https://github.com/anoma/juvix/issues/1714
A lambda:
```
\ { v0 := b0 ; v1 := b1 ; ... ; vn := bn }
```
should be translated to:
```
λ? (λ? ... (λ? (match ?$0, ?$1, ... , ?$n with ...)))
```
i.e the de Brujin index of the values in the match always start from 0.
Fixes: https://github.com/anoma/juvix/issues/1695
Adds Juvix tests for the compilation pipeline - these are converted from
the JuvixCore tests (those that make sense). Currently, only the
translation from Juvix to JuvixCore is checked for the tests that can be
type-checked. Ultimately, the entire compilation pipeline down to native
code / WebAssembly should be checked on these tests.
Closes#1689
Closes#1483
The error now points to the offending `=`, works correctly with
multi-line clauses, and explains exactly what's wrong. E.g. for
```agda
f : Nat → Nat → Nat;
f zero x = x;
```
we get
```
|
6 | f zero x = x;
| ^
expected ":=" instead of "="
```
A minor disadvantage of the proposed solution is that now it's
impossible to use `=` without parentheses as a top variable name in a
pattern, e.g.
```
f zero = := =;
```
gives an error.
However, if one really wants to name a variable `=`, it is enough just
to enclose it in parentheses:
```agda
f : Nat → Nat → Nat;
f zero (=) := =;
f (suc n) (=) := f n =;
```
I believe this slight non-uniformity is well worth the increased
usability due to a better error message. Confusing `:=` with `=` is very
common. Using `=` as a variable name in a top pattern is rare.
Co-authored-by: janmasrovira <janmasrovira@gmail.com>
This PR adds smoke tests using [Smoke
tool](https://github.com/SamirTalwar/smoke) for all the shell tests we
have. One reason for adopting Smoke instead of the previous tool,
`shelltestrunner`, is that tests are declared cleanly and simply using
Smoke Yaml syntax compared to shelltestrunner's syntax.
To add a new smoke test, create a file with the suffix ".smoke.yaml" in
the `tests/smoke` folder. In such a folder, you can also find examples
of how to test the CLI.
An implementation of the translation from JuvixCore to JuvixAsm. After
merging this PR, the only remaining step to complete the basic
compilation pipeline (#1556) is the compilation of complex pattern
matching (#1531).
* Fixes several bugs in lambda-lifting.
* Fixes several bugs in the RemoveTypeArgs transformation.
* Fixes several bugs in the TopEtaExpand transformation.
* Adds the ConvertBuiltinTypes transformation which converts the builtin
bool inductive type to Core primitive bool.
* Adds the `juvix dev core strip` command.
* Adds the `juvix dev core asm` command.
* Adds the `juvix dev core compile` command.
* Adds two groups of tests:
- JuvixCore to JuvixAsm translation: translate JuvixCore tests to
JuvixAsm and run the results with the JuvixAsm interpreter,
- JuvixCore compilation: compile JuvixCore tests to native code and WASM
and execute the results.
* Closes#1520
* Closes#1549
* Fixes#1678.
* Adds the `clean-juvix-build` Makefile target, which removes all
`.juvix-build` directories in the project (necessary to do after
changing the standard library).
* Depends on PR #1688. The tests go through without merging this PR, but
it's a bug. The present PR requires the possibility to use the
`terminating` keyword with the `div` built-in, which possibility is
provided by PR #1688.
