This patch dramatically increases the efficiency of `juvix dev root`,
which was unnecessarily parsing all dependencies included in the
`Package.juvix` file. Other commands that do not require the `Package`
will also be faster.
It also refactors some functions so that the `TaggedLock` effect is run
globally.
I've added `singletons-base` as a dependency so we can have `++` on the
type level. We've tried to define a type family ourselves but inference
was not working properly.
## Overview
This PR makes the compiler pipeline thread-safe so that the test suite
can be run in parallel.
This is achieved by:
* Removing use of `{get, set, with}CurrentDir` functions.
* Adding locking around shared file resources like the the
global-project and internal build directory.
NB: **Locking is disabled for the main compiler target**, as it is
single threaded they are not required.
## Run test suite in parallel
To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to
your stack test arguments. For example:
```
stack test --fast --ta '+RTS -N -RTS'
```
The `-N` instructs the Haskell runtime to choose the number of threads
to use based on how many processors there are on your machine. You can
use `-Nn` to see the number of threads to `n`.
These flags are already [set in the
Makefile](e6dca22cfd/Makefile (L26))
when you or CI uses `stack test`.
## Locking
The Haskell package
[filelock](https://hackage.haskell.org/package/filelock) is used for
locking. File locks are used instead of MVars because Juvix code does
not control when new threads are created, they are created by the test
suite. This means that MVars created by Juvix code will have no effect,
because they are created independently on each test-suite thread.
Additionally the resources we're locking live on the filesystem and so
can be conveniently tagged by path.
### FileLock
The filelock library is wrapped in a FileLock effect:
e6dca22cfd/src/Juvix/Data/Effect/FileLock/Base.hs (L6-L8)
There is an [IO
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/IO.hs (L8))
that uses filelock and an [no-op
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/Permissive.hs (L7))
that just runs actions unconditionally.
### TaggedLock
To make the file locks simpler to use a TaggedLock effect is introduced:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock/Base.hs (L5-L11)
And convenience function:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock.hs (L28)
This allows an action to be locked, tagged by a directory that may or
may not exist. For example in the following code, an action is performed
on a directory `root` that may delete the directory before repopulating
the files. So the lockfile cannot be stored in the `root` itself.
e6dca22cfd/src/Juvix/Extra/Files.hs (L55-L60)
## Pipeline
As noted above, we only use locking in the test suite. The main app
target pipeline is single threaded and so locking is unnecessary. So the
interpretation of locks is parameterised so that locking can be disabled
e6dca22cfd/src/Juvix/Compiler/Pipeline/Run.hs (L64)
Previously if you call Juvix on a file that doesn't exist you get the
error:
```
$ juvix compile /i/don't/exist.juvix
juvix: /i/dont: changeWorkingDirectory: does not exist (No such file or directory)
```
After this change you will see:
```
$ juvix compile /i/don't/exist.juvix
The input path "/i/dont/exist.juvix" does not exist
```
- Closes#1993
This pr makes it possible to use `~`, `..` and environment variables in
the `juvix.yaml` and all flags / input of the cli.
In the CLI, the shell will be responsible for replacing environment
variables with their value, so the usual syntax can be used. For the
`dependencies` field, I have implemented a parser that has some
restrictions:
1. Environment variables are given with the makefile-like syntax
`$(VAR)`
2. The three characters `$` `(` `)` are reserved for the environment
variables syntax.
They cannot be part of the path.
3. `~` is reserved for `$(HOME)`. I.e. the prepath `~~` will expand to
`$HOME$HOME`.
4. Nested environment variables are not allowed.
Thanks @paulcadman for the feedback. I think we are ready to merge this
nightmarish pr 👻
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
This PR adds `juvix format` that can be used to format either a single
Juvix file or all files in a Juvix project.
## Usage
```
$ juvix format --help
Usage: juvix format JUVIX_FILE_OR_PROJECT [--check] [--in-place]
Format a Juvix file or Juvix project
When the command is run with an unformatted file it prints the reformatted source to standard output.
When the command is run with a project directory it prints a list of unformatted files in the project.
Available options:
JUVIX_FILE_OR_PROJECT Path to a .juvix file or to a directory containing a
Juvix project.
--check Do not print reformatted sources or unformatted file
paths to standard output.
--in-place Do not print reformatted sources to standard output.
Overwrite the target's contents with the formatted
version if the formatted version differs from the
original content.
-h,--help Show this help text
```
## Location of main implementation
The implementation is split into two components:
* The src API: `format` and `formatProject`
73952ba15c/src/Juvix/Formatter.hs
* The CLI interface:
73952ba15c/app/Commands/Format.hs
## in-place uses polysemy Resource effect
The `--in-place` option makes a backup of the target file and restores
it if there's an error during processing to avoid data loss. The
implementation of this uses the polysemy [Resource
effect](https://hackage.haskell.org/package/polysemy-1.9.0.0/docs/Polysemy-Resource.html).
The recommended way to interpret the resource effect is to use
`resourceToIOFinal` which makes it necessary to change the effects
interpretation in main to use `Final IO`:
73952ba15c/app/Main.hs (L15)
## Format input is `FilePath`
The format options uses `FilePath` instead of `AppFile f` for the input
file/directory used by other commands. This is because we cannot
determine if the input string is a file or directory in the CLI parser
(we require IO). I discussed some ideas with @janmasrovira on how to
improve this in a way that would also solve other issues with CLI input
file/parsing but I want to defer this to a separate PR as this one is
already quite large.
One consequence of Format using `FilePath` as the input option is that
the code that changes the working directory to the root of the project
containing the CLI input file is changed to work with `FilePath`:
f715ef6a53/app/TopCommand/Options.hs (L33)
## New dependencies
This PR adds new dependencies on `temporary` and `polysemy-zoo`.
`temporary` is used for `emptySystemTempFile` in the implementation of
the TempFile interpreter for IO:
73952ba15c/src/Juvix/Data/Effect/Files/IO.hs (L49)
`polysemy-zoo` is used for the `Fresh` effect and `absorbMonadThrow` in
the implementation of the pure TempFile interpreter:
73952ba15c/src/Juvix/Data/Effect/Files/Pure.hs (L91)
NB: The pure TempFile interpreter is not used, but it seemed a good idea
to include it while it's fresh in my mind.
* Closes https://github.com/anoma/juvix/issues/1777
---------
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
* Add shelltests for `juvix repl`
* repl: Compute entrypoint root when loading a file
Previously the REPL would use app root (which could be the current
directory or the project root of the initially loaded file). Thus files
in a different project could not be loaded.
The entrypoint root must be computed each time a new file is `:load`ed.
Adds shell-tests for REPL commands
* Move preludePath to Juvix.Extra.Paths
* w.i.p
* Added strict positivity condition for datatypes w.i.p
* Add negative test for str.postivity check
* Add some revisions
* the branch is back to feet
* w.i.p add lots of traces to check alg.
* Add more test and revisions
* Add negative and positive test to the new flag and the keyword
* Fix shell tests.
* Make pre-commit happy
* Fix rebase conflicts
* Make pre-commit happy
* Add shell test, rename keyword, fix error msg
* Revert change
* Necessary changes
* Remove wrong unless
* Move the positivity checker to its own folder
* Add missing juvix.yaml
* Add a negative test thanks to jan
* make some style changes
* Make ormolu happy
* Remove unnecessary instance of Show
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
* Renaming MiniJuvix to Juvix
* Make Ormolu happy
* Make Hlint happy
* Remove redundant imports
* Fix shell tests and add target ci to our Makefile
* Make pre-commit happy
* Embed stdlib in minijuvix library
We add a new step at the beginning of the pipeline called Setup that
registers the modules in the standard library with the Files effect. The
standard library is then used when the Scoper queries the Files effect
for modules as it resolves import statements.
Use of the standard library can be disabled using the global
`--no-stdlib` command-line option.
* CI: Checkout submodules recursively for stdlib
* Add a new `--no-stdlib` option to shell check
* Poke CI
* CI: Checkout submodules in the test job
* work in progress towards implicit arguments
* Wip towards implicit types
* improve arity checker
* Add version of SimpleFungibleToken with implicit arguments
* guess arity of body before checking the lhs of a clause
* add ArityUnknown and fix some tests
* wip: proper errors in arity checker
* fix bugs, improve errors and add tests
* format
* set hlint version to 3.4 in the ci
* update pre-commit version to 3.0.0
* minor changes
* added more revisions
* minor
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
* Remove input file fields from command opts
* [cli] Make version and help commands
* Fix on reviews
* Fixes for dealing with global options inside subcmds
* Fix minijuvix emacs mode and add some instance to GlobalOpts
* Remove unrelated code
* Propagate globals opts in each cmd parser
* Add initial shell tests
* Add test-shell to makefile and CI
* Fix CI: adding .local/bin to PATH
* Fixing CI
* Installing shelltest just before running it
* Install app for shell testing
* Hide global flags after cmd. Fix shell tests accordingly.
* Fixing CI
* Shell test only run on ubuntu for now
* [app] Add compile command
The compile command drives the (external) clang compiler to build a WASM
binary from a minijuvix file.
The intermediate C files are stored in .minijuvix-build directory
relative to the directory where the command is executed.
The user can specify which runtime (standalone and libc) to build
against.
The file-embed library is used to embed the runtime files into the
minijuvix binary.
* TemplateHaskell is a default extension
* [app] Add the .minijuvix-build dir in the root of the project
* [app] Derive the default output filename from the inputfile
* [chore] Add .minijuvix-build to .gitignore
* [WIP] EntryPoint now has options. --no-termination is a new global opt.
* Add TerminationChecking to the pipeline
* Add TerminationChecking to the pipeline
* Keep GlobalOptions in App
* Fix reviewer's comments
* delete unnecessary parens
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
* add face and handling of not in scope symbol error
* small fix
* generic errors wip
* add App effect
* format
* add flycheck-minijuvix
* use absolute paths and refactor
* fix dir0
* add generic error instances and improve some errors
* format
* qualify strings
* use AnsiText
* add ToGenericError instances for the type checker errors
* improve error message
* improve handling of parsing errors
* [cbackend] Adds an AST for C
This should cover enough C to implement the microjuvix backend.
* [cbackend] Add C serializer using language-c library
We may decide to write our own serializer for the C AST but this
demonstrates that the C AST is sufficient at least.
* [cbackend] Declarations will always be typed
* [cbackend] Add CPP support to AST
* [cbackend] Rename some names for clarity
* [cbackend] Add translation of InductiveDef to C
* [cbackend] Add CLI for C backend
* [cbackend] Add stdbool.h to file header
* [cbackend] Allow Cpp and Verbatim code inline
* [cbackend] Add a newline after printing C
* [cbackend] Support foreign blocks
* [cbackend] Add support for axioms
* [cbackend] Remove code examples
* [cbackend] wip FunctionDef including Expressions
* [parser] Support esacping '}' inside a foreign block
* [cbackend] Add support for patterns in functions
* [cbackend] Add foreign C support to HelloWorld.mjuvix
* hlint fixes
* More hlint fixes not picked up by pre-commit
* [cbackend] Remove CompileStatement from MonoJuvix
* [cbackend] Add support for compile blocks
* [cbackend] Move compileInfo extraction to MonoJuvixResult
* [minihaskell] Fix compile block support
* [chore] Remove ununsed isBackendSupported function
* [chore] Remove unused imports
* [cbackend] Use a Reader for pattern bindings
* [cbackend] Fix compiler warnings
* [cbackend] Add support for nested patterns
* [cbackend] Use functions to instantiate argument names
* [cbackend] Add non-exhaustive pattern error message
* [cbackend] Adds test for c to WASM compile and execution
* [cbackend] Add links to test dependencies in quickstart
* [cbackend] Add test with inductive types and patterns
* [cbackend] Fix indentation
* [cbackend] Remove ExpressionTyped case
https://github.com/heliaxdev/minijuvix/issues/79
* [lexer] Fix lexing of \ inside a foreign block
* [cbackend] PR review fixes
* [chore] Remove unused import
* [cbackend] Rename CJuvix to MiniC
* [cbackend] Rename MonoJuvixToC to MonoJuvixToMiniC
* [cbackend] Add test for polymorphic function
* [cbackend] Add module for string literals
* Parsing terminating keyword
* cosmetics
* Add support for `terminating` keyword to function decl.
* Minor replacement
* Fix .mjuxix to .mjuvix and the module name
* add TypeCallsBuilder and others
* implement propagation of type calls
* improve type propagation
* polymorphize fungible token
* sort type calls map pretty output
* use HashSet in TypeCallsMap
* renaming
* rename module
* improve indexing in type propagation
* draft monomorphization generation algorithm
* fix draft
* wip mono code generation
* wip code generation
* finish first candidate for code generation
* add monojuvix command
* fix MonoJuvix pretty printer to properly display name ids
* [monojuvix] improve clause pretty printing
* add support for function types in expressions
* properly translate function expressions
* ormolu
* add a basic positive test for monomorphization
* cleanup effect constraints
* collect type applications in axiom types
* apply some style improvements
* fix PolySimpleFungibleToken and add it to the test suite
* ignore polymorphic inductive definitions that are never used
