Implements a disassembler which converts Cairo bytecode into textual
CASM representation. Useful for debugging the Cairo backend.
* Adds the `juvix dev casm from-cairo` command
Each commit in this PR is a separate improvement.
* Tag any Term with a string instead of just cells using `@`. e.g
`"myTag" @ opCall ...`
* `:dump FILE` in the nockma REPL to dump the last REPL result to a
file.
* More tagging in the pretty nockma output.
# Changes
The main goal of this pr is to remove the `--target` flag for `juvix
compile` and use subcommands instead. The targets that are relevant to
normal users are found in `juvix compile --help`. Targets that are
relevant only to developers are found in `juvix dev compile --help`.
Below I list some of the changes in more detail.
## Compile targets for user-facing languages
- `juvix compile native`
- `juvix compile wasi`. I wasn't sure how to call this: `wasm`,
`wasm32-wasi`, etc. In the end I thought `wasi` was short and accurate,
but we can change it.
- `juvix compile vampir`
- `juvix compile anoma`
- `juvix compile cairo`
## *New* compile targets for internal languages
See `juvix dev compile --help`.
1. `dev compile core` has the same behaviour as `dev core
from-concrete`. The `dev core from-concrete` is redundant at the moment.
2. `dev compile tree` compiles to Tree and prints the InfoTable to the
output file wihout any additional checks.
3. `dev compile reg` compiles to Reg and prints the InfoTable to the
output file wihout any additional checks.
4. `dev compile asm` compiles to Asm and prints the InfoTable to the
output file wihout any additional checks.
5. 4. `dev compile casm` compiles to Asm and prints the Result to the
output file wihout any additional checks. TODO: should the Result be
printed or something else? At the moment the Result lacks a pretty
instance.
6.
## Optional input file
1. The input file for commands that expect a .juvix file as input is now
optional. If the argument is ommited, he main file given in the
package.yaml will be used. This applies to the following commands:
1. `juvix compile [native|wasi|geb|vampir|anoma|cairo]`
8. `juvix dev compile [core|reg|tree|casm|asm]`
1. `juvix html`
3. `juvix markdown`.
4. `juvix dev internal [typecheck|pretty]`.
5. `juvix dev [parse|scope]`
7. `juvix compile [native|wasi|geb|vampir|anoma|cairo]`
9. note that `juvix format` has not changed its behaviour.
## Refactor some C-like compiler flags
Both `juvix compile native` and `juvix compile wasi` support `--only-c`
(`-C`), `--only-preprocess` (`-E`), `--only-assemble` (`-S`). I propose
to deviate from the `gcc` style and instead use a flag with a single
argument:
- `--cstage [source|preprocess|assembly|exec(default)]`. I'm open to
suggestions. For now, I've kept the legacy flags but marked them as
deprecated in the help message.
## Remove code duplication
I've tried to reduce code duplication. This is sometimes in tension with
code readability so I've tried to find a good balance. I've tried to
make it so we don't have to jump to many different files to understand
what a single command is doing. I'm sure there is still room for
improvement.
## Other refactors
I've implemented other small refactors that I considered improved the
quality of the code.
## TODO/Future work
We should refactor commands (under `compile dev`) which still use
`module Commands.Extra.Compile` and remove it.
* Closes#2687
* Adds hint support in CASM. The supported hints are `Input(var)` and
`Alloc(size)`. These are the hints currently implemented in
[juvix-cairo-vm](https://github.com/anoma/juvix-cairo-vm).
* Adds the `--program_input` option to the `juvix dev casm run` command.
* Enables private inputs via `main` arguments. In generated CASM/Cairo
code, the arguments to `main` are fetched using the `Input` hint.
* Modifies the CI to use
[juvix-cairo-vm](https://github.com/anoma/juvix-cairo-vm)
* Closes#2563
Checklist
------------
- [x] Serialization of the Haskell CASM representation to the JSON
format accepted by the Cairo VM.
- [x] Add the `cairo` target to the `compile` commands.
- [x] Output via the Cairo `output` builtin.
- [x] Relativize jumps. Cairo VM doesn't actually support absolute
jumps.
- [x] Test the translation from CASM to Cairo by running the output in
the Cairo VM
- [x] Add Cairo VM to the CI
When we first implemented the Nockma backend we wrongly assumed that the
only entry point for Juvix compiled Nockma modules would be the main
function. Using this assumption we could add a setup step in the main
function that put the Anoma stdlib and compiled functions from the Juvix
module in a static place in the Nockma subject. References to the Anoma
stdlib and functions in the module could then be resolved statically.
However, one of the use cases for Juvix -> Nockma compilation is for
Anoma to run logic functions that are fields of a transaction. So the
user writes a Juvix program with main function that returns a
transaction. The result of the main function is passed to Anoma. When
Anoma calls the logic function on a field of the transaction, the setup
part of the main function is not run so the subject is not in the
required state. In fact, the logic function is not even callable by
Anoma because non-main functions in the Juvix module use a calling
convention that assumes the subject has a particular shape.
This PR solves the problem by making all functions in the Juvix module
use the Anoma calling convention. We make all compiled closures
(including, for example, the logic functions stored on resources in a
transaction) self contained, i.e they contain the functions library and
anoma standard library.
Modules that contain many closures produce large nockma output files
which slows down the evaluator. This will need to be fixed in the future
either with Nockma compression ([jam
serialization](https://developers.urbit.org/reference/hoon/stdlib/2p))
or otherwise. But it does not block the compilation and execution of
Anoma transactions.
Other fixes / additions:
* Extra tracing. You can now annotate output cells with a tag that will
be displayed in the output
* Unittests for listToTuple, appendRights helper functions
* Fixes for the nockma parser when parsing 'pretty nockma', specifically
stdlib calls, tags and functions_library atom.
* Adds `juvix dev nock run` command that can run a program output with
the `anoma` target.
* Remove the `nockma` target. As described above we always use the Anoma
calling convention so there's no need for a separate target for the
'juvix calling convention'
* Adds a `--profile` flag to `juvix dev nock run` which outputs a count
of Nockma ops used in the evaluation
* In tests we no longer serialise the compiled program to force full
evaluation of the compiled code. We added a negative test to check that
strings are not allowed in Nockma/Anoma programs,
it is output in a file `OUTPUT.profile` and has the following form:
```
quote : 15077
apply : 0
isCell : 0
suc : 0
= : 4517
if : 5086
seq : 5086
push : 0
call : 4896
replace : 1
hint : 8
scry : 0
trace : 0
```
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
The following benchmark compares juvix 0.6.0 with polysemy and a new
version (implemented in this pr) which replaces polysemy by effectful.
# Typecheck standard library without caching
```
hyperfine --warmup 2 --prepare 'juvix-polysemy clean' 'juvix-polysemy typecheck Stdlib/Prelude.juvix' 'juvix-effectful typecheck Stdlib/Prelude.juvix'
Benchmark 1: juvix-polysemy typecheck Stdlib/Prelude.juvix
Time (mean ± σ): 3.924 s ± 0.143 s [User: 3.787 s, System: 0.084 s]
Range (min … max): 3.649 s … 4.142 s 10 runs
Benchmark 2: juvix-effectful typecheck Stdlib/Prelude.juvix
Time (mean ± σ): 2.558 s ± 0.074 s [User: 2.430 s, System: 0.084 s]
Range (min … max): 2.403 s … 2.646 s 10 runs
Summary
juvix-effectful typecheck Stdlib/Prelude.juvix ran
1.53 ± 0.07 times faster than juvix-polysemy typecheck Stdlib/Prelude.juvix
```
# Typecheck standard library with caching
```
hyperfine --warmup 1 'juvix-effectful typecheck Stdlib/Prelude.juvix' 'juvix-polysemy typecheck Stdlib/Prelude.juvix' --min-runs 20
Benchmark 1: juvix-effectful typecheck Stdlib/Prelude.juvix
Time (mean ± σ): 1.194 s ± 0.068 s [User: 0.979 s, System: 0.211 s]
Range (min … max): 1.113 s … 1.307 s 20 runs
Benchmark 2: juvix-polysemy typecheck Stdlib/Prelude.juvix
Time (mean ± σ): 1.237 s ± 0.083 s [User: 0.997 s, System: 0.231 s]
Range (min … max): 1.061 s … 1.476 s 20 runs
Summary
juvix-effectful typecheck Stdlib/Prelude.juvix ran
1.04 ± 0.09 times faster than juvix-polysemy typecheck Stdlib/Prelude.juvix
```
* Closes#2562
Checklist
---------
- [x] Translation from JuvixReg to CASM
- [x] CASM runtime
- [x] Juvix to CASM pipeline: combine the right transformations and
check prerequisites
- [x] CLI commands: add target `casm` to the `compile` commands
- [x] Tests:
- [x] Test the translation from JuvixReg to CASM
- [x] Test the entire pipeline from Juvix to CASM
This PR adds some flags to tune the (html/md) output.
For Markdown subcommand:
```
--strip-prefix ARG Strip the given prefix from the input file path for
HTML hyperlinks
--folder-structure Generate HTML following the module's folder structure
```
For HTML subcommand, we have the ones above plus the following:
```
--ext ARG File extension in hyperlinks for the input file
(default: ".html")
```
* Closes#2571
* It is reasonable to finish this PR before tackling #2562, because the
field element type is the primary data type in Cairo.
* Depends on #2653
Checklist
---------
- [x] Add field type and operations to intermediate representations
(JuvixCore, JuvixTree, JuvixAsm, JuvixReg).
- [x] Add CLI option to choose field size.
- [x] Add frontend field builtins.
- [x] Automatic conversion of integer literals to field elements.
- [x] Juvix standard library support for fields.
- [x] Check if field size matches when loading a stored module.
- [x] Update the Cairo Assembly (CASM) interpreter to use the field type
instead of integer type.
- [x] Add field type to VampIR backend.
- [x] Tests
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
Builtin information needs to be propagated from stored modules to REPL
artifacts to avoid "The builtin _ has not been defined" errors.
This PR adds a test suite for the REPL in the Haskell test code. This
means some of the slow smoke tests can be moved to fast haskell unit
tests. In future we should refactor the REPL code by putting in the main
src target and unit testing more features (e.g :doc, :def).
* Closes https://github.com/anoma/juvix/issues/2638
This PR adds a `anoma` target to the `juvix compile`. This target
compiles a Juvix `main` function to a Nockma/Anoma "function". Unlike
the native, wasm, and nockma targets the main function may have any type
signature.
## Anoma calling convention
[Anoma calls
functions](6a4e15fe9c/lib/anoma/resource.ex (L122))
by evaluating the formula `[call L replace [RL args] @ S]` against a
subject equal to the function. Here `args` is a Nockma term that
evaluates to a tuple of arguments that should be passed to the function.
The anoma target compiles the `main` function to Nockma in the same way
as the nockma target. The main function is then
[wrapped](9a658465ae/src/Juvix/Compiler/Nockma/Translation/FromTree.hs (L627))
to make it compatible with the Anoma calling convention.
## Testing
The anoma calling convention is [unit
tested](9a658465ae/test/Nockma/Eval/Positive.hs (L117))
and [smoke
tested](9a658465ae/tests/smoke/Commands/compile.smoke.yaml (L159)).
This PR also adds versions of the end-to-end compilation tests. Most
tests are included, tests for builtin IO operations and string builtins
are omitted. Other tests that use Strings have been adapted to use other
types that are compatible with this backend.
## Nockma REPL
To facilitate testing the Nockma REPL can now load a nockma file as an
initial subject.
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
Now the prelude exports this function:
```
readFile :: (MonadIO m) => Path Abs File -> m Text
readFile = liftIO . Utf8.readFile . toFilePath
```
It is more convenient to use because it uses typed `Path` and works in
any `MonadIO`.
* Implements JuvixReg recursors, which will allow to implement JuvixReg
transformations more succinctly and effectively.
* Adds a transformation framework to JuvixReg.
* Adds identity transformation tests.
* Depends on #2635
- ⚠️ Depends on #2644
The `effectful` library does not support the `Embed` effect out of the
box. However, it offers `IOE`, which is equivalent to `Embed IO` from
polysemy. In preparation to a possible migration to `effectful`, this pr
hides the general `Embed` effect from the prelude and it exports a
specialized `EmbedIO` in its place.
Since we upgraded to ghc-9.8.1 and dropped the dependency on
[with-utf8](https://hackage.haskell.org/package/with-utf8), the function
`writeFileEnsureLn` no longer has the constraint `MonadMask m`, so the
`embed @IO` has become redundant
* Closes#2578
* Implements JuvixReg parser and pretty printer.
* Adds the `juvix dev reg read file.jvr` command.
* Adds the `reg` target to the `compile` commands.
* Adds tests for the JuvixReg parser.
This allows us to do `juvix dev tree compile --target nockma test.jvt`.
The pipeline was already implemented, this pr only adds the
`TargetNockma` to the list of supported backends so that the cli
recognizes it.
This pr implements two additional versions of the Juvix Tree evaluator.
Now we have
1. The raw implementation that does not use effects. It throws Haskell
exceptions for errors. It uses `unsafePerformIO` for traces. It relies
on bang patterns to force strictness and guarantee the expected order of
execution (for traces). Avoiding effects allows for improved execution
efficiency.
2. [`polysemy`](https://hackage.haskell.org/package/polysemy-1.9.1.3)
based implementation.
3.
[`effectful-core`](https://hackage.haskell.org/package/effectful-core)
based implementation.
One can specify which evaluator to use thus:
```
juvix dev tree eval --evaluator XXX test.jvt
```
where XXX is one of `raw`, `polysemy`, `effectful`.
# Preliminary benchmarks
More thorough benchmarks should be run, but here are some preliminary
results:
## Test032 (Fibonacci 20)
I've adapted test032 so that it main is a single call to fibonacci of
20.
Command:
```
hyperfine --warmup 2 --runs 10 'juvix dev tree eval test032.jvt --evaluator polysemy' 'juvix dev tree eval test032.jvt --evaluator raw' 'juvix dev tree eval test032.jvt --evaluator e
ffectful'
```
Output:
```
Benchmark 1: juvix dev tree eval test032.jvt --evaluator polysemy
Time (mean ± σ): 2.133 s ± 0.040 s [User: 2.113 s, System: 0.016 s]
Range (min … max): 2.088 s … 2.227 s 10 runs
Benchmark 2: juvix dev tree eval test032.jvt --evaluator raw
Time (mean ± σ): 308.7 ms ± 13.8 ms [User: 293.6 ms, System: 14.1 ms]
Range (min … max): 286.5 ms … 330.1 ms 10 runs
Benchmark 3: juvix dev tree eval test032.jvt --evaluator effectful
Time (mean ± σ): 366.0 ms ± 2.8 ms [User: 345.4 ms, System: 19.4 ms]
Range (min … max): 362.5 ms … 372.6 ms 10 runs
Summary
juvix dev tree eval test032.jvt --evaluator raw ran
1.19 ± 0.05 times faster than juvix dev tree eval test032.jvt --evaluator effectful
6.91 ± 0.34 times faster than juvix dev tree eval test032.jvt --evaluator polysemy
```
## Test034 (Higher-order function composition)
A modified version of test034 where main defined as `call[exp](3, 12)`
Command:
```
hyperfine --warmup 2 --runs 10 'juvix dev tree eval test034.jvt --evaluator polysemy' 'juvix dev tree eval test034.jvt --evaluator raw' 'juvix dev tree eval test034.jvt --evaluator effectful'
```
Output:
```
Benchmark 1: juvix dev tree eval test034.jvt --evaluator polysemy
Time (mean ± σ): 7.025 s ± 0.184 s [User: 6.518 s, System: 0.469 s]
Range (min … max): 6.866 s … 7.327 s 10 runs
Benchmark 2: juvix dev tree eval test034.jvt --evaluator raw
Time (mean ± σ): 835.6 ms ± 7.4 ms [User: 757.2 ms, System: 75.9 ms]
Range (min … max): 824.7 ms … 847.4 ms 10 runs
Benchmark 3: juvix dev tree eval test034.jvt --evaluator effectful
Time (mean ± σ): 1.578 s ± 0.010 s [User: 1.427 s, System: 0.143 s]
Range (min … max): 1.563 s … 1.595 s 10 runs
Summary
juvix dev tree eval test034.jvt --evaluator raw ran
1.89 ± 0.02 times faster than juvix dev tree eval test034.jvt --evaluator effectful
8.41 ± 0.23 times faster than juvix dev tree eval test034.jvt --evaluator polysemy
```
## Test036 (Streams without memoization)
A modified version of test036 where main defined as `call[nth](700,
call[primes]())`
Command:
```
hyperfine --warmup 2 --runs 5 'juvix dev tree eval test036.jvt --evaluator polysemy' 'juvix dev tree eval test036.jvt --evaluator raw' 'juvix dev tree eval test036.jvt --evaluator effectful'
```
Output:
```
Benchmark 1: juvix dev tree eval test036.jvt --evaluator polysemy
Time (mean ± σ): 1.993 s ± 0.026 s [User: 1.946 s, System: 0.043 s]
Range (min … max): 1.969 s … 2.023 s 5 runs
Benchmark 2: juvix dev tree eval test036.jvt --evaluator raw
Time (mean ± σ): 137.5 ms ± 7.1 ms [User: 127.5 ms, System: 8.9 ms]
Range (min … max): 132.8 ms … 149.8 ms 5 runs
Benchmark 3: juvix dev tree eval test036.jvt --evaluator effectful
Time (mean ± σ): 329.0 ms ± 7.3 ms [User: 289.3 ms, System: 37.4 ms]
Range (min … max): 319.9 ms … 336.0 ms 5 runs
Summary
juvix dev tree eval test036.jvt --evaluator raw ran
2.39 ± 0.13 times faster than juvix dev tree eval test036.jvt --evaluator effectful
14.50 ± 0.77 times faster than juvix dev tree eval test036.jvt --evaluator polysemy
```
This PR updates the stackage LTS resolver to `nightly-2024-02-06` which
uses GHC 9.8.1
## Upgrade notes
You will need to update your HLS to
[2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0),
this release contains support for GHC 9.8.1
## Fixes
### `haskeline` / `repline`
We have removed the custom haskeline / repline forks used in the build.
This is because we had trouble overriding haskeline as it is bundled
with GHC and the stackage resolver uses this bundled version. We were
using a custom fork of haskeline to implement
[mapInputT_](15c0685c91/app/Commands/Repl.hs (L409))
in the Juvix REPL, required to implement error handling. This requires
private API from the Haskeline library.
Instead of using a custom fork we use TemplateHaskell to obtain access
to the private API we need. See
[DarkArts.hs](15c0685c91/src/Juvix/Prelude/DarkArts.hs)
and
[HaskelineJB.hs](15c0685c91/app/HaskelineJH.hs).
To obtain access to the private API, we adapted a method from [a Tweag
blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/)
and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC
9.8.1.
### `aeson-better-errors`
The `aeson-better-errors` library has not been updated to work with
`mtl-2.3.0` so it cannot work with the new stackage resolver. We are
using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which
has been updated.
We should consider replacing this library in future, see
https://github.com/anoma/juvix/issues/2621
### `path`
The `path` library now includes API `splitDrive` and `dropDrive` so we
can remove our versions of those functions from the prelude.
### `with-utf8`
We no longer need to depend on `with-utf8`. We were using this package
for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are
now available in the `text` package.
### Compiler warnings
GHC 9.8.1 introduces several new compiler warnings.
* We have suppressed `missing-role-annotations` and
`missing-poly-kind-signatures`
* We added our own versions of `head` and `tail` to work around the new
`partial-tx` warning introduced for those functions in `Data.List`.
* We fixed up the code to avoid the
[term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture)
warning.
This PR replaces the JuvixAsm -> Nockma translation with a JuvixTree ->
Nockma translation.
We can now enable some of the JuvixTree tests that did not work with
JuvixAsm->Nockma because they were too slow.
## Notes
We have changed [test031: temp stack with
branching](22ee87f0e7/tests/Tree/positive/test031.jvt)
to avoid using negative numbers (because negative integers are not
supported in Nockma).
Three tree tests trace/output lists. Lists are serialised differently by
the asm and nockma pretty printers so they cannot share a single test
output file. We have created separate nockma output files for these
tests (see eg.
[test028.nockma.out](22ee87f0e7/tests/Tree/positive/out/test028.nockma.out)).
* Closes https://github.com/anoma/juvix/issues/2606
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
* Adds the `--transforms`, `--eval` and `--no-print` options to the
`juvix dev tree read` command.
* Depends on #2598
* Depends on #2597
* Depends on #2596
* Depends on #2595
* Depends on #2594
* Depends on #2590
Now we guarantee that whenever we write a file there is a newline
character at the end, which is a [Unix
convention](https://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap03.html#tag_03_205).
The juvix prelude now exports `writeFileEnsureLn` and it no longer
exports `writeFile`. If at some point we need the behaviour of
`writeFile` I'd suggest that we export it renamed as `writeFileVerbatim`
or something similar.
* Moves the "apply" transformation from JuivxAsm to JuvixTree. This
transformation removes the `CallClosures` nodes.
* Makes Nockma compilation tests use JuvixTree instead of JuvixAsm
files.
* Depends on #2594
* Depends on #2590
* Depends on #2589
* Depends on #2587
* Implements a translation from JuvixAsm to JuvixTree. It does not work
in general, but works for all code generated from Juvix and all JuvixAsm
tests.
* Adds the `juvix dev tree from-asm` command.
* Adds tests automatically converted from JuvixAsm tests.
* Depends on #2583
This PR implements:
* JuvixTree parser.
* JuvixTree pretty printer.
* `juvix dev tree read file.jvt` command which reads and pretty prints a
JuvixTree file.
* The `tree` target in the `compile` command.
* Removal of `StackRef` in JuvixAsm. This makes JuvixAsm more consistent
with JuvixTree and simplifies the data structures. `StackRef` is not
needed for compilation from Core.
Tests for the parser will appear in a separate PR, when I implement an
automatic translation of JuvixAsm to JuvixTree files.
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
Adds annotations to cells to indicate that it is a call to the stdlib
and might be evaluated faster in the Haskell evaluator.
The syntax for stdlib calls is as follows:
```
[stdlib@add args@<args-term> <left-term> <right-term>]
```
where `add` is the name of the function being called, `<args-term>` is a
nockma term that points to the position of the arguments, and
`<left-term>` and `<right-term>` are the actual components of the cell.
This PR:
* introduces the JuvixTree language which is like JuvixAsm except that
instead of the value stack there is an applicative structure,
* refactors the JuvixCore -> JuvixAsm translation into JuvixCore ->
JuvixTree -> JuvixAsm.
JuvixAsm is a bit too low level for efficient compilation to Nock.
Translating the value stack explicitly is a bad idea and it's
unnecessary, because the value stack just represents an applicative
structure which can be represented directly in Nock. It's possible, but
cumbersome and unnecessary, to recover the applicative structure from
JuvixAsm code. It's better to have a bit more high-level JuvixTree
language which still retains the explicit applicative structure.
This PR is a snapshot of the current work on the JuvixAsm -> Nockma
translation. The compilation of Juvix programs to Nockma now works so we
decided to raise this PR now to avoid it getting too large.
## Juvix -> Nockma compilation
You can compile a frontend Juvix file to Nockma as follows:
example.juvix
```
module example;
import Stdlib.Prelude open;
fib : Nat → Nat → Nat → Nat
| zero x1 _ := x1
| (suc n) x1 x2 := fib n x2 (x1 + x2);
fibonacci (n : Nat) : Nat := fib n 0 1;
sumList (xs : List Nat) : Nat :=
for (acc := 0) (x in xs)
acc + x;
main : Nat := fibonacci 9 + sumList [1; 2; 3; 4];
```
```
$ juvix compile -t nockma example.juvix
```
This will generate a file `example.nockma` which can be run using the
nockma evaluator:
```
$ juvix dev nockma eval example.nockma
```
Alternatively you can compile JuvixAsm to Nockma:
```
$ juvix dev asm compile -t nockma example.jva
```
## Tests
We compile an evaluate the JuvixAsm tests in
cb3659e08e/test/Nockma/Compile/Asm/Positive.hs
We currently skip some because either:
1. They are too slow to run in the current evaluator (due to arithmetic
operations using the unjetted nock code from the anoma nock stdlib).
2. They trace data types like lists and booleans which are represented
differently by the asm interpreter and the nock interpreter
3. They operate on raw negative numbers, nock only supports raw natural
numbers
## Next steps
On top of this PR we will work on improving the evaluator so that we can
enable the slow compilation tests.
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
This PR contains refactors split out from the Nockma compile PR
https://github.com/anoma/juvix/pull/2570. Each refactor is associated
with a separate commit in this PR.
* Closes#2561
* Defines an extended subset of Cairo Assembly, following Section 5 of
[1].
* Adds the commands `juvix dev casm read file.casm` and `juvix dev casm
run file.casm` to print and run `*.casm` files.
* The tests cover CASM semantics. Some are "manual translations" of
corresponding JuvixAsm tests according to the JuvixAsm -> CASM
compilation concept.
This PR adds an parser, pretty printer, evaluator, repl and quasi-quoter
for Nock terms.
## Parser / Pretty Printer
The parser and pretty printer handle both standard Nock terms and
'pretty' Nock terms (where op codes and paths can be named). Standard
and pretty Nock forms can be mixed in the same term.
For example instead of `[0 2]` you can write `[@ L]`.
See
a6028b0d92/src/Juvix/Compiler/Nockma/Language.hs (L79)
for the correspondence between pretty Nock and Nock operators.
In pretty Nock, paths are represented as strings of `L` (for head) and
`R` (for tail) instead of the number encoding in standard nock. The
character `S` is used to refer to the whole subject, i.e it is sugar for
`1` in standard Nock.
See
a6028b0d92/src/Juvix/Compiler/Nockma/Language.hs (L177)
for the correspondence between pretty Nock path and standard Nock
position.
## Quasi-quoter
A quasi-quoter is added so Nock terms can be included in the source, e.g
`[nock| [@ LL] |]`.
## REPL
Launch the repl with `juvix dev nockma repl`.
A Nock `[subject formula]` cell is input as `subject / formula` , e.g:
```
nockma> [1 0] / [@ L]
1
```
The subject can be set using `:set-stack`.
```
nockma> :set-stack [1 0]
nockma> [@ L]
1
```
The subject can be viewed using `:get-stack`.
```
nockma> :set-stack [1 0]
nockma> :get-stack
[1 0]
```
You can assign a Nock term to a variable and use it in another
expression:
```
nockma> r := [@ L]
nockma> [1 0] / r
1
```
A list of assignments can be read from a file:
```
$ cat stack.nock
r := [@ L]
$ juvix dev nockma repl
nockma> :load stack.nock
nockma> [1 0] / r
1
```
* Closes https://github.com/anoma/juvix/issues/2557
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
* Closes#2392
Changes checklist
-----------------
* [X] Abstract out data types for stored module representation
(`ModuleInfo` in `Juvix.Compiler.Store.Language`)
* [X] Adapt the parser to operate per-module
* [X] Adapt the scoper to operate per-module
* [X] Adapt the arity checker to operate per-module
* [X] Adapt the type checker to operate per-module
* [x] Adapt Core transformations to operate per-module
* [X] Adapt the pipeline functions in `Juvix.Compiler.Pipeline`
* [X] Add `Juvix.Compiler.Pipeline.Driver` which drives the per-module
compilation process
* [x] Implement module saving / loading in `Pipeline.Driver`
* [x] Detect cyclic module dependencies in `Pipeline.Driver`
* [x] Cache visited modules in memory in `Pipeline.Driver` to avoid
excessive disk operations and repeated hash re-computations
* [x] Recompile a module if one of its dependencies needs recompilation
and contains functions that are always inlined.
* [x] Fix identifier dependencies for mutual block creation in
`Internal.fromConcrete`
- Fixed by making textually later definitions depend on earlier ones.
- Now instances are used for resolution only after the textual point of
their definition.
- Similarly, type synonyms will be unfolded only after the textual point
of their definition.
* [x] Fix CLI
* [x] Fix REPL
* [x] Fix highlighting
* [x] Fix HTML generation
* [x] Adapt test suite
This patch dramatically increases the efficiency of `juvix dev root`,
which was unnecessarily parsing all dependencies included in the
`Package.juvix` file. Other commands that do not require the `Package`
will also be faster.
It also refactors some functions so that the `TaggedLock` effect is run
globally.
I've added `singletons-base` as a dependency so we can have `++` on the
type level. We've tried to define a type family ourselves but inference
was not working properly.
This PR creates a new package that's bundled with the compiler in a
similar way to the stdlib and the package description package.
## The `package-base` Package
This package is called
[package-base](ab4376cf9e/include/package-base)
and contains the minimal set of definitions required to load a Package
file.
The
[`Juvix.Builtin`](ab4376cf9e/include/package-base/Juvix/Builtin/V1.juvix)
module contains:
```
module Juvix.Builtin.V1;
import Juvix.Builtin.V1.Nat open public;
import Juvix.Builtin.V1.Trait.Natural open public;
import Juvix.Builtin.V1.String open public;
import Juvix.Builtin.V1.Bool open public;
import Juvix.Builtin.V1.Maybe open public;
import Juvix.Builtin.V1.List open public;
import Juvix.Builtin.V1.Fixity open public;
```
`Juvix.Builtin.V1.Bool` is required to support backend primitive
integers `Juvix.Builtin.V1.Trait.Natural` is required to support numeric
literals.
## The `PackageDescription.V2` module
This PR also adds a new
[`PackageDescription.V2`](ab4376cf9e/include/package/PackageDescription/V2.juvix)
type that uses the `package-base`. This is to avoid breaking existing
Package files. The Packages files in the repo (except those that test
`PackageDescription.V1`) have also been updated.
## Updating the stdlib
The standard library will be updated to use `Juvix.Builtin.*` modules in
a subsequent PR.
* Part of https://github.com/anoma/juvix/issues/2511
## Overview
This PR makes the compiler pipeline thread-safe so that the test suite
can be run in parallel.
This is achieved by:
* Removing use of `{get, set, with}CurrentDir` functions.
* Adding locking around shared file resources like the the
global-project and internal build directory.
NB: **Locking is disabled for the main compiler target**, as it is
single threaded they are not required.
## Run test suite in parallel
To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to
your stack test arguments. For example:
```
stack test --fast --ta '+RTS -N -RTS'
```
The `-N` instructs the Haskell runtime to choose the number of threads
to use based on how many processors there are on your machine. You can
use `-Nn` to see the number of threads to `n`.
These flags are already [set in the
Makefile](e6dca22cfd/Makefile (L26))
when you or CI uses `stack test`.
## Locking
The Haskell package
[filelock](https://hackage.haskell.org/package/filelock) is used for
locking. File locks are used instead of MVars because Juvix code does
not control when new threads are created, they are created by the test
suite. This means that MVars created by Juvix code will have no effect,
because they are created independently on each test-suite thread.
Additionally the resources we're locking live on the filesystem and so
can be conveniently tagged by path.
### FileLock
The filelock library is wrapped in a FileLock effect:
e6dca22cfd/src/Juvix/Data/Effect/FileLock/Base.hs (L6-L8)
There is an [IO
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/IO.hs (L8))
that uses filelock and an [no-op
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/Permissive.hs (L7))
that just runs actions unconditionally.
### TaggedLock
To make the file locks simpler to use a TaggedLock effect is introduced:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock/Base.hs (L5-L11)
And convenience function:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock.hs (L28)
This allows an action to be locked, tagged by a directory that may or
may not exist. For example in the following code, an action is performed
on a directory `root` that may delete the directory before repopulating
the files. So the lockfile cannot be stored in the `root` itself.
e6dca22cfd/src/Juvix/Extra/Files.hs (L55-L60)
## Pipeline
As noted above, we only use locking in the test suite. The main app
target pipeline is single threaded and so locking is unnecessary. So the
interpretation of locks is parameterised so that locking can be disabled
e6dca22cfd/src/Juvix/Compiler/Pipeline/Run.hs (L64)
