Builtin information needs to be propagated from stored modules to REPL
artifacts to avoid "The builtin _ has not been defined" errors.
This PR adds a test suite for the REPL in the Haskell test code. This
means some of the slow smoke tests can be moved to fast haskell unit
tests. In future we should refactor the REPL code by putting in the main
src target and unit testing more features (e.g :doc, :def).
* Closes https://github.com/anoma/juvix/issues/2638
This PR adds a `anoma` target to the `juvix compile`. This target
compiles a Juvix `main` function to a Nockma/Anoma "function". Unlike
the native, wasm, and nockma targets the main function may have any type
signature.
## Anoma calling convention
[Anoma calls
functions](6a4e15fe9c/lib/anoma/resource.ex (L122))
by evaluating the formula `[call L replace [RL args] @ S]` against a
subject equal to the function. Here `args` is a Nockma term that
evaluates to a tuple of arguments that should be passed to the function.
The anoma target compiles the `main` function to Nockma in the same way
as the nockma target. The main function is then
[wrapped](9a658465ae/src/Juvix/Compiler/Nockma/Translation/FromTree.hs (L627))
to make it compatible with the Anoma calling convention.
## Testing
The anoma calling convention is [unit
tested](9a658465ae/test/Nockma/Eval/Positive.hs (L117))
and [smoke
tested](9a658465ae/tests/smoke/Commands/compile.smoke.yaml (L159)).
This PR also adds versions of the end-to-end compilation tests. Most
tests are included, tests for builtin IO operations and string builtins
are omitted. Other tests that use Strings have been adapted to use other
types that are compatible with this backend.
## Nockma REPL
To facilitate testing the Nockma REPL can now load a nockma file as an
initial subject.
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
* Closes#2578
* Implements JuvixReg parser and pretty printer.
* Adds the `juvix dev reg read file.jvr` command.
* Adds the `reg` target to the `compile` commands.
* Adds tests for the JuvixReg parser.
* Implements a translation from JuvixAsm to JuvixTree. It does not work
in general, but works for all code generated from Juvix and all JuvixAsm
tests.
* Adds the `juvix dev tree from-asm` command.
* Adds tests automatically converted from JuvixAsm tests.
* Depends on #2583
* Closes#2561
* Defines an extended subset of Cairo Assembly, following Section 5 of
[1].
* Adds the commands `juvix dev casm read file.casm` and `juvix dev casm
run file.casm` to print and run `*.casm` files.
* The tests cover CASM semantics. Some are "manual translations" of
corresponding JuvixAsm tests according to the JuvixAsm -> CASM
compilation concept.
This PR adds an parser, pretty printer, evaluator, repl and quasi-quoter
for Nock terms.
## Parser / Pretty Printer
The parser and pretty printer handle both standard Nock terms and
'pretty' Nock terms (where op codes and paths can be named). Standard
and pretty Nock forms can be mixed in the same term.
For example instead of `[0 2]` you can write `[@ L]`.
See
a6028b0d92/src/Juvix/Compiler/Nockma/Language.hs (L79)
for the correspondence between pretty Nock and Nock operators.
In pretty Nock, paths are represented as strings of `L` (for head) and
`R` (for tail) instead of the number encoding in standard nock. The
character `S` is used to refer to the whole subject, i.e it is sugar for
`1` in standard Nock.
See
a6028b0d92/src/Juvix/Compiler/Nockma/Language.hs (L177)
for the correspondence between pretty Nock path and standard Nock
position.
## Quasi-quoter
A quasi-quoter is added so Nock terms can be included in the source, e.g
`[nock| [@ LL] |]`.
## REPL
Launch the repl with `juvix dev nockma repl`.
A Nock `[subject formula]` cell is input as `subject / formula` , e.g:
```
nockma> [1 0] / [@ L]
1
```
The subject can be set using `:set-stack`.
```
nockma> :set-stack [1 0]
nockma> [@ L]
1
```
The subject can be viewed using `:get-stack`.
```
nockma> :set-stack [1 0]
nockma> :get-stack
[1 0]
```
You can assign a Nock term to a variable and use it in another
expression:
```
nockma> r := [@ L]
nockma> [1 0] / r
1
```
A list of assignments can be read from a file:
```
$ cat stack.nock
r := [@ L]
$ juvix dev nockma repl
nockma> :load stack.nock
nockma> [1 0] / r
1
```
* Closes https://github.com/anoma/juvix/issues/2557
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
* Closes#2392
Changes checklist
-----------------
* [X] Abstract out data types for stored module representation
(`ModuleInfo` in `Juvix.Compiler.Store.Language`)
* [X] Adapt the parser to operate per-module
* [X] Adapt the scoper to operate per-module
* [X] Adapt the arity checker to operate per-module
* [X] Adapt the type checker to operate per-module
* [x] Adapt Core transformations to operate per-module
* [X] Adapt the pipeline functions in `Juvix.Compiler.Pipeline`
* [X] Add `Juvix.Compiler.Pipeline.Driver` which drives the per-module
compilation process
* [x] Implement module saving / loading in `Pipeline.Driver`
* [x] Detect cyclic module dependencies in `Pipeline.Driver`
* [x] Cache visited modules in memory in `Pipeline.Driver` to avoid
excessive disk operations and repeated hash re-computations
* [x] Recompile a module if one of its dependencies needs recompilation
and contains functions that are always inlined.
* [x] Fix identifier dependencies for mutual block creation in
`Internal.fromConcrete`
- Fixed by making textually later definitions depend on earlier ones.
- Now instances are used for resolution only after the textual point of
their definition.
- Similarly, type synonyms will be unfolded only after the textual point
of their definition.
* [x] Fix CLI
* [x] Fix REPL
* [x] Fix highlighting
* [x] Fix HTML generation
* [x] Adapt test suite
## Overview
This PR makes the compiler pipeline thread-safe so that the test suite
can be run in parallel.
This is achieved by:
* Removing use of `{get, set, with}CurrentDir` functions.
* Adding locking around shared file resources like the the
global-project and internal build directory.
NB: **Locking is disabled for the main compiler target**, as it is
single threaded they are not required.
## Run test suite in parallel
To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to
your stack test arguments. For example:
```
stack test --fast --ta '+RTS -N -RTS'
```
The `-N` instructs the Haskell runtime to choose the number of threads
to use based on how many processors there are on your machine. You can
use `-Nn` to see the number of threads to `n`.
These flags are already [set in the
Makefile](e6dca22cfd/Makefile (L26))
when you or CI uses `stack test`.
## Locking
The Haskell package
[filelock](https://hackage.haskell.org/package/filelock) is used for
locking. File locks are used instead of MVars because Juvix code does
not control when new threads are created, they are created by the test
suite. This means that MVars created by Juvix code will have no effect,
because they are created independently on each test-suite thread.
Additionally the resources we're locking live on the filesystem and so
can be conveniently tagged by path.
### FileLock
The filelock library is wrapped in a FileLock effect:
e6dca22cfd/src/Juvix/Data/Effect/FileLock/Base.hs (L6-L8)
There is an [IO
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/IO.hs (L8))
that uses filelock and an [no-op
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/Permissive.hs (L7))
that just runs actions unconditionally.
### TaggedLock
To make the file locks simpler to use a TaggedLock effect is introduced:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock/Base.hs (L5-L11)
And convenience function:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock.hs (L28)
This allows an action to be locked, tagged by a directory that may or
may not exist. For example in the following code, an action is performed
on a directory `root` that may delete the directory before repopulating
the files. So the lockfile cannot be stored in the `root` itself.
e6dca22cfd/src/Juvix/Extra/Files.hs (L55-L60)
## Pipeline
As noted above, we only use locking in the test suite. The main app
target pipeline is single threaded and so locking is unnecessary. So the
interpretation of locks is parameterised so that locking can be disabled
e6dca22cfd/src/Juvix/Compiler/Pipeline/Run.hs (L64)
* Closes#2035
* Depends on #2086
* Depends on #2096
* Adds end-to-end tests for the Juvix-to-VampIR compilation pipeline.
---------
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
This PR adds `juvix format` that can be used to format either a single
Juvix file or all files in a Juvix project.
## Usage
```
$ juvix format --help
Usage: juvix format JUVIX_FILE_OR_PROJECT [--check] [--in-place]
Format a Juvix file or Juvix project
When the command is run with an unformatted file it prints the reformatted source to standard output.
When the command is run with a project directory it prints a list of unformatted files in the project.
Available options:
JUVIX_FILE_OR_PROJECT Path to a .juvix file or to a directory containing a
Juvix project.
--check Do not print reformatted sources or unformatted file
paths to standard output.
--in-place Do not print reformatted sources to standard output.
Overwrite the target's contents with the formatted
version if the formatted version differs from the
original content.
-h,--help Show this help text
```
## Location of main implementation
The implementation is split into two components:
* The src API: `format` and `formatProject`
73952ba15c/src/Juvix/Formatter.hs
* The CLI interface:
73952ba15c/app/Commands/Format.hs
## in-place uses polysemy Resource effect
The `--in-place` option makes a backup of the target file and restores
it if there's an error during processing to avoid data loss. The
implementation of this uses the polysemy [Resource
effect](https://hackage.haskell.org/package/polysemy-1.9.0.0/docs/Polysemy-Resource.html).
The recommended way to interpret the resource effect is to use
`resourceToIOFinal` which makes it necessary to change the effects
interpretation in main to use `Final IO`:
73952ba15c/app/Main.hs (L15)
## Format input is `FilePath`
The format options uses `FilePath` instead of `AppFile f` for the input
file/directory used by other commands. This is because we cannot
determine if the input string is a file or directory in the CLI parser
(we require IO). I discussed some ideas with @janmasrovira on how to
improve this in a way that would also solve other issues with CLI input
file/parsing but I want to defer this to a separate PR as this one is
already quite large.
One consequence of Format using `FilePath` as the input option is that
the code that changes the working directory to the root of the project
containing the CLI input file is changed to work with `FilePath`:
f715ef6a53/app/TopCommand/Options.hs (L33)
## New dependencies
This PR adds new dependencies on `temporary` and `polysemy-zoo`.
`temporary` is used for `emptySystemTempFile` in the implementation of
the TempFile interpreter for IO:
73952ba15c/src/Juvix/Data/Effect/Files/IO.hs (L49)
`polysemy-zoo` is used for the `Fresh` effect and `absorbMonadThrow` in
the implementation of the pure TempFile interpreter:
73952ba15c/src/Juvix/Data/Effect/Files/Pure.hs (L91)
NB: The pure TempFile interpreter is not used, but it seemed a good idea
to include it while it's fresh in my mind.
* Closes https://github.com/anoma/juvix/issues/1777
---------
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
In this PR I will add tests for the example programs in
`examples/milestone`.
There's currently an runtime assertion error generated by the Hanoi
example https://github.com/anoma/juvix/issues/1919, so it'd be good to
test these programs in the future.
* Depends on PR #1824
* Closes#1556
* Closes#1825
* Closes#1843
* Closes#1729
* Closes#1596
* Closes#1343
* Closes#1382
* Closes#1867
* Closes#1876
* Changes the `juvix compile` command to use the new pipeline.
* Removes the `juvix dev minic` command and the `BackendC` tests.
* Adds the `juvix eval` command.
* Fixes bugs in the Nat-to-integer conversion.
* Fixes bugs in the Internal-to-Core and Core-to-Core.Stripped
translations.
* Fixes bugs in the RemoveTypeArgs transformation.
* Fixes bugs in lambda-lifting (incorrect de Bruijn indices in the types
of added binders).
* Fixes several other bugs in the compilation pipeline.
* Adds a separate EtaExpandApps transformation to avoid quadratic
runtime in the Internal-to-Core translation due to repeated calls to
etaExpandApps.
* Changes Internal-to-Core to avoid generating matches on values which
don't have an inductive type.
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
Co-authored-by: janmasrovira <janmasrovira@gmail.com>
This PR introduces an evaluator for the Geb STLC interface/fragment and
other related commands, including a REPL to interact with his backend.
-
https://github.com/anoma/geb/blob/mariari/binaries/src/specs/lambda.lisp
We have included a REPL and support for commands such as read and eval
here. Check out:
```
juvix dev geb --help
```
- [x] Add Geb evaluator with the two basic eval strategies.
- [x] Add quasi quoter: return morphisms from typed geb values.
- [x] Add type/object inference for morphisms.
- [x] All combined: morphisms-eval-to-morphisms
- [x] Parse and pretty printer Geb values (without quoting them)
- [x] Parse files containing Geb terms:
- [x] Saved in a .lisp file according to anoma/geb example (typed
object).
- [x] Store in a .geb file simple as simple lisp expression.
- [x] Add related commands to the CLI for `dev geb`:
- [x] Subcommand: eval
- [x] Subcommand: read
- [x] Subcommand: infer
- [x] Subcommand: repl
- [x] Subcommand: check
- [x] Minor changes `hom` by `!->` in the Geb prettyprinter
- [x] Add tests for:
- [x] New subcommand (smoke tests)
- [x] Eval
Issues to solve after merging this PR:
- Add location to Geb ast for proper error location.
- Add tests for all related subcommands, e.g. check, and infer.
- Check compilation from Core to Geb: (run inferObject with the type
provided by the core node).
- [x] Update the vs code-plugin to load Geb repl and eval.
(31994c8684)
Adds Juvix tests for the compilation pipeline - these are converted from
the JuvixCore tests (those that make sense). Currently, only the
translation from Juvix to JuvixCore is checked for the tests that can be
type-checked. Ultimately, the entire compilation pipeline down to native
code / WebAssembly should be checked on these tests.
Closes#1689
* Compute name dependency graph and filter unreachable declarations
* bugfix: recurse into type signatures
* positive tests
* make ormolu happy
* get starting nodes from ExportInfo
* make ormolu happy
* cosmetic refactoring of DependencyInfo
* fix tests & style
* Renaming MiniJuvix to Juvix
* Make Ormolu happy
* Make Hlint happy
* Remove redundant imports
* Fix shell tests and add target ci to our Makefile
* Make pre-commit happy
* work in progress towards implicit arguments
* Wip towards implicit types
* improve arity checker
* Add version of SimpleFungibleToken with implicit arguments
* guess arity of body before checking the lhs of a clause
* add ArityUnknown and fix some tests
* wip: proper errors in arity checker
* fix bugs, improve errors and add tests
* format
* set hlint version to 3.4 in the ci
* update pre-commit version to 3.0.0
* minor changes
* added more revisions
* minor
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
* [WIP] EntryPoint now has options. --no-termination is a new global opt.
* Add TerminationChecking to the pipeline
* Add TerminationChecking to the pipeline
* Keep GlobalOptions in App
* Fix reviewer's comments
* delete unnecessary parens
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
* [cbackend] Adds an AST for C
This should cover enough C to implement the microjuvix backend.
* [cbackend] Add C serializer using language-c library
We may decide to write our own serializer for the C AST but this
demonstrates that the C AST is sufficient at least.
* [cbackend] Declarations will always be typed
* [cbackend] Add CPP support to AST
* [cbackend] Rename some names for clarity
* [cbackend] Add translation of InductiveDef to C
* [cbackend] Add CLI for C backend
* [cbackend] Add stdbool.h to file header
* [cbackend] Allow Cpp and Verbatim code inline
* [cbackend] Add a newline after printing C
* [cbackend] Support foreign blocks
* [cbackend] Add support for axioms
* [cbackend] Remove code examples
* [cbackend] wip FunctionDef including Expressions
* [parser] Support esacping '}' inside a foreign block
* [cbackend] Add support for patterns in functions
* [cbackend] Add foreign C support to HelloWorld.mjuvix
* hlint fixes
* More hlint fixes not picked up by pre-commit
* [cbackend] Remove CompileStatement from MonoJuvix
* [cbackend] Add support for compile blocks
* [cbackend] Move compileInfo extraction to MonoJuvixResult
* [minihaskell] Fix compile block support
* [chore] Remove ununsed isBackendSupported function
* [chore] Remove unused imports
* [cbackend] Use a Reader for pattern bindings
* [cbackend] Fix compiler warnings
* [cbackend] Add support for nested patterns
* [cbackend] Use functions to instantiate argument names
* [cbackend] Add non-exhaustive pattern error message
* [cbackend] Adds test for c to WASM compile and execution
* [cbackend] Add links to test dependencies in quickstart
* [cbackend] Add test with inductive types and patterns
* [cbackend] Fix indentation
* [cbackend] Remove ExpressionTyped case
https://github.com/heliaxdev/minijuvix/issues/79
* [lexer] Fix lexing of \ inside a foreign block
* [cbackend] PR review fixes
* [chore] Remove unused import
* [cbackend] Rename CJuvix to MiniC
* [cbackend] Rename MonoJuvixToC to MonoJuvixToMiniC
* [cbackend] Add test for polymorphic function
* [cbackend] Add module for string literals
* add TypeCallsBuilder and others
* implement propagation of type calls
* improve type propagation
* polymorphize fungible token
* sort type calls map pretty output
* use HashSet in TypeCallsMap
* renaming
* rename module
* improve indexing in type propagation
* draft monomorphization generation algorithm
* fix draft
* wip mono code generation
* wip code generation
* finish first candidate for code generation
* add monojuvix command
* fix MonoJuvix pretty printer to properly display name ids
* [monojuvix] improve clause pretty printing
* add support for function types in expressions
* properly translate function expressions
* ormolu
* add a basic positive test for monomorphization
* cleanup effect constraints
* collect type applications in axiom types
* apply some style improvements
* fix PolySimpleFungibleToken and add it to the test suite
* ignore polymorphic inductive definitions that are never used
* [ format ] AbstractToMicroJuvix
* [ CI ] fixes
* [ CI ] fixes
* [ CI ] Using GHC 9.0 for Hlint
* [ CI ] Use static-checks for Dev as well
* [test] Add positive test for typechecker
* [test] Improve positive typechecker error output
* [typecheck] Restore correct handling of TypeAny
I mistakenly removed the matchTypes function in
https://github.com/heliaxdev/minijuvix/pull/22. This caused the handling
of TypeAny to break.
Literals have type TypeAny and so should be valid when matching against
any other type. The tests have been updated to reflect this.
* [test] Add positive MicroJuvix typecheck tests
* [ ormolu ] fixes
Co-authored-by: Jonathan Prieto-Cubides <jonathan.cubides@uib.no>
