Adds a new version of the lock file that stores the hash (sha256 digest)
of the package file (Package.juvix, juvix.yaml) it was generated from as
a field:
```
# This file was autogenerated by Juvix version 0.5.4.
# Do not edit this file manually.
version: 2
checksum: d05940a4d3dc0e15451d02e1294819c875ba486ee54e26865ba8d190ac7c27c3
dependencies:
- git:
name: stdlib
ref: f68b0614ad695eaa13ead42f3466e0a78219f826
url: https://github.com/anoma/juvix-stdlib.git
dependencies: []
```
The lock file is regenerated when the hash of the package file doesn't
match the value of the `checksum` field, i.e when the user updates the
package file.
Existing lock files are automatically migrated to version 2.
* Closes https://github.com/anoma/juvix/issues/2464
## Overview
This PR makes the compiler pipeline thread-safe so that the test suite
can be run in parallel.
This is achieved by:
* Removing use of `{get, set, with}CurrentDir` functions.
* Adding locking around shared file resources like the the
global-project and internal build directory.
NB: **Locking is disabled for the main compiler target**, as it is
single threaded they are not required.
## Run test suite in parallel
To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to
your stack test arguments. For example:
```
stack test --fast --ta '+RTS -N -RTS'
```
The `-N` instructs the Haskell runtime to choose the number of threads
to use based on how many processors there are on your machine. You can
use `-Nn` to see the number of threads to `n`.
These flags are already [set in the
Makefile](e6dca22cfd/Makefile (L26))
when you or CI uses `stack test`.
## Locking
The Haskell package
[filelock](https://hackage.haskell.org/package/filelock) is used for
locking. File locks are used instead of MVars because Juvix code does
not control when new threads are created, they are created by the test
suite. This means that MVars created by Juvix code will have no effect,
because they are created independently on each test-suite thread.
Additionally the resources we're locking live on the filesystem and so
can be conveniently tagged by path.
### FileLock
The filelock library is wrapped in a FileLock effect:
e6dca22cfd/src/Juvix/Data/Effect/FileLock/Base.hs (L6-L8)
There is an [IO
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/IO.hs (L8))
that uses filelock and an [no-op
interpreter](e6dca22cfd/src/Juvix/Data/Effect/FileLock/Permissive.hs (L7))
that just runs actions unconditionally.
### TaggedLock
To make the file locks simpler to use a TaggedLock effect is introduced:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock/Base.hs (L5-L11)
And convenience function:
e6dca22cfd/src/Juvix/Data/Effect/TaggedLock.hs (L28)
This allows an action to be locked, tagged by a directory that may or
may not exist. For example in the following code, an action is performed
on a directory `root` that may delete the directory before repopulating
the files. So the lockfile cannot be stored in the `root` itself.
e6dca22cfd/src/Juvix/Extra/Files.hs (L55-L60)
## Pipeline
As noted above, we only use locking in the test suite. The main app
target pipeline is single threaded and so locking is unnecessary. So the
interpretation of locks is parameterised so that locking can be disabled
e6dca22cfd/src/Juvix/Compiler/Pipeline/Run.hs (L64)
* Closes#2365
* Implements the syntax `f@{x1 := def1; ...; xn := defn}` and `f@?{x1 :=
def1; ..; xn := defn}`. Each definition inside the `@{..}` is an
ordinary function definition. The `@?` version allows partial
application (not all explicit named arguments need to be provided). This
subsumes the old record creation syntax.
Depends on:
* ~~https://github.com/anoma/juvix/pull/2459~~
* https://github.com/anoma/juvix/pull/2462
This PR is part of a series implementing:
* https://github.com/anoma/juvix/issues/2336
This PR adds the package file loading function, including a file
evaluation effect. It integrates this with the existing `readPackage`
function and adds tests / smoke tests.
## Package.juvix format
Instead of `juvix.yaml` (which is still supported currently) users can
now place a `Package.juvix` file in the root of their project. The
simplest `Package.juvix` file you can write is:
```
module Package;
import PackageDescription open;
package : Package := defaultPackage;
```
The
[PackageDescription](35b2f618f0/include/package/PackageDescription.juvix)
module defines the `Package` type. Users can use "go-to definition" in
their IDE from the Package file to see the documentation and
definitions.
Users may also import `Stdlib.Prelude` in their Package file. This is
loaded from the global project. No other module imports are supported.
Notes:
* If a directory contains both `Package.juvix` and `juvix.yaml` then
`Package.juvix` is used in preference.
## Default stdlib dependency
The `Dependency` type has a constructor called `defaultStdlib`. This
means that any project can use the compiler builtin standard library
dependency. With `juvix.yaml` this dependency is only available when the
`dependencies` field is unspecified.
```
module Package;
import PackageDescription open;
package : Package := defaultPackage { dependencies := [defaultStdlib] };
```
## Validation
As well as the standard type checking validation that the Juvix compiler
provides additional validation is made on the file.
* The Package module must contain the identifier `package` and it must
have type `Package` that's obtained from the global `PackageDescription`
module.
* Every dependency specified in the Package.juvix must be unique.
* Closes https://github.com/anoma/juvix/issues/2336
## Examples
### Package with name and version
```
module Package;
import PackageDescription open;
package : Package :=
defaultPackage {name := "a-package";
version := mkVersion 0 1 0};
```
### Package with GitHub dependency
```
module Package;
import PackageDescription open;
package : Package :=
defaultPackage {name := "a-package";
version := mkVersion 0 1 0;
dependencies := [defaultStdlib;
github (org := "anoma";
repo := "juvix-containers";
ref := "v0.7.1")]};
```
## Package with main and buildDir fields
```
module Package;
import Stdlib.Prelude open;
import PackageDescription open;
package : Package :=
defaultPackage {name := "a-package";
version := mkVersion 0 1 0;
dependencies := [defaultStdlib;
github (org := "anoma";
repo := "juvix-containers";
ref := "v0.7.1")];
buildDir := just "/tmp/build";
main := just "HelloWorld.juvix"
};
```
The special PathResolver puts files from the global package stdlib and
files from the global package description files in scope of the
$root/Package.juvix module.
Currently this means that PackageDescription module is in scope for the
module so that the user can write:
```
module Package;
import Stdlib.Prelude open;
import PackageDescription open;
package : Package :=
mkPackageDefault
(name := "foo")
{ version := mkVersion 0 1 0
; dependencies :=
[ github "anoma" "juvix-stdlib" "adf58a7180b361a022fb53c22ad9e5274ebf6f66"
; github "anoma" "juvix-containers" "v0.7.1"]};
```
This PR adds a new command `juvix dependencies update` that fetches all
dependencies in a project and updates the project lock file.
Currently the only way to update the lock file is to delete it and
generate a new one.
## CLI Docs
```
juvix dependencies --help
Usage: juvix dependencies COMMAND
Subcommands related to dependencies
Available options:
-h,--help Show this help text
Available commands:
update Fetch package dependencies and update the lock file
```
## Example
A project containing the following `juvix.yaml`
```yaml
dependencies:
- .juvix-build/stdlib/
- git:
url: https://github.com/anoma/juvix-test
ref: v0.6.0
name: test
main: Example.juvix
name: example
version: 1.0.0
```
compile to generate the lockfile: `juvix compile`
```yaml
# This file was autogenerated by Juvix version 0.5.1.
# Do not edit this file manually.
dependencies:
- path: .juvix-build/stdlib/
dependencies: []
- git:
name: test
ref: a94c61749678ff57556ee6e4cb1f8fbbddbc4ab1
url: https://github.com/anoma/juvix-test
dependencies:
- git:
name: stdlib
ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
url: https://github.com/anoma/juvix-stdlib
dependencies: []
```
Now update the test dependency version:
```yaml
- .juvix-build/stdlib/
- git:
url: https://github.com/anoma/juvix-test
ref: v0.6.1
name: test
main: Example.juvix
name: example
version: 1.0.0
```
And run `juvix dependencies update`
Now the lockfile has updated to the hash of v0.6.1:
```yaml
# This file was autogenerated by Juvix version 0.5.1.
# Do not edit this file manually.
dependencies:
- path: .juvix-build/stdlib/
dependencies: []
- git:
name: test
ref: a7ac74cac0db92e0b5e349f279d797c3788cdfdd
url: https://github.com/anoma/juvix-test
dependencies:
- git:
name: stdlib
ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
url: https://github.com/anoma/juvix-stdlib
dependencies: []
```
---------
Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
This PR adds lock file support to the compiler pipeline. The lock file
is generated whenever a compiler pipeline command (`juvix {compile,
typecheck, repl}`) is run.
The lock file contains all the information necessary to reproduce the
whole dependency source tree. In particular for git dependencies,
branch/tag references are resolved to git hash references.
## Lock file format
The lock file is a YAML `juvix.lock.yaml` file written by the compiler
alongside the package's `juvix.yaml` file.
```
LOCKFILE_SPEC: { dependencies: { DEPENDENCY_SPEC, dependencies: LOCKFILE_SPEC }
DEPENDENCY_SPEC: PATH_SPEC | GIT_SPEC
PATH_SPEC: { path: String }
GIT_SPEC: { git: {url: String, ref: String, name: String } }
```
## Example
Consider a project containing the following `juvix.yaml`:
```yaml
dependencies:
- .juvix-build/stdlib/
- git:
url: https://github.com/anoma/juvix-containers
ref: v0.7.1
name: containers
name: example
version: 1.0.0
```
After running `juvix compile` the following lockfile `juvix.lock.yaml`
is generated.
```yaml
# This file was autogenerated by Juvix version 0.5.1.
# Do not edit this file manually.
dependencies:
- path: .juvix-build/stdlib/
dependencies: []
- git:
name: containers
ref: 3debbc7f5776924eb9652731b3c1982a2ee0ff24
url: https://github.com/anoma/juvix-containers
dependencies:
- git:
name: stdlib
ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
url: https://github.com/anoma/juvix-stdlib
dependencies: []
- git:
name: test
ref: a7ac74cac0db92e0b5e349f279d797c3788cdfdd
url: https://github.com/anoma/juvix-test
dependencies:
- git:
name: stdlib
ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
url: https://github.com/anoma/juvix-stdlib
dependencies: []
```
For subsequent runs of the juvix compile pipeline, the lock file
dependency information is used.
## Behaviour when package file and lock file are out of sync
If a dependency is specified in `juvix.yaml` that is not present in the
lock file, an error is raised.
Continuing the example above, say we add an additional dependency:
```
dependencies:
- .juvix-build/stdlib/
- git:
url: https://github.com/anoma/juvix-containers
ref: v0.7.1
name: containers
- git:
url: https://github.com/anoma/juvix-test
ref: v0.6.1
name: test
name: example
version: 1.0.0
```
`juvix compile` will throw an error:
```
/Users/paul/tmp/lockfile/dep/juvix.yaml:1:1: error:
The dependency test is declared in the package's juvix.yaml but is not declared in the lockfile: /Users/paul/tmp/lockfile/dep/juvix.lock.json
Try removing /Users/paul/tmp/lockfile/dep/juvix.lock.yaml and then run Juvix again.
```
Closes:
* https://github.com/anoma/juvix/issues/2334
* Closes#2280
* Record creation syntax uses normal function definition syntax like at
the top-level or in lets.
* It is now allowed to omit the result type annotation in function
definitions (the `: ResultType` part) with `_` inserted by default. This
is allowed only for simple definitions of the form `x := value` in lets
and record creation, but not at the top level.
- Closes#2330
- Closes#2329
This pr implements the syntax changes described in #2330. It drops
support for the old yaml-based syntax.
Some valid examples:
```
syntax iterator for {init := 1; range := 1};
syntax fixity cons := binary {assoc := right};
syntax fixity cmp := binary;
syntax fixity cmp := binary {}; -- debatable whether we want to accept empty {} or not. I think we should
```
# Future work
This pr creates an asymmetry between iterators and operators
definitions. Iterators definition do not require a constructor. We could
add it to make it homogeneous, but it looks a bit redundant:
```
syntax iterator for := mkIterator {init := 1; range := 1};
```
We could consider merging iterator and fixity declarations with this
alternative syntax.
```
syntax XXX for := iterator {init := 1; range := 1};
syntax XXX cons := binary {assoc := right};
```
where `XXX` is a common keyword. Suggestion by @lukaszcz XXX = declare
---------
Co-authored-by: Łukasz Czajka <62751+lukaszcz@users.noreply.github.com>
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
This pr simplifies parsing by removing `FunctionParameterUnnamed`. It
also removes ghost wildcards introduced during parsing.
It also introduces an error for double braced atoms `{{x}}` that are not
on the left of an arrow `->`
This PR introduces a global `--offline` flag.
## Doctor
This replaces the `--offline` flag on the doctor command.
## Juvix package builds
The flag applies to juvix build commands like `juvix compile`, `juvix
repl`. This is so that users can continue to build packages offline that
have external dependencies when there's no network connection (as long
as they built the same package online previously).
Specifically, when the `--offline` flag is used in a package that has
external git dependencies.
* No `git clone` or `git fetch` commands are used
* `git checkout` will continue to be used
* Clones from previous builds are reused
This means that you can update the `ref` field in a git dependency, as
long as the ref existed the last time that the project was built without
the `--offline` flag.
* Closes https://github.com/anoma/juvix/issues/2333
This PR adds external git dependency support to the Juvix package
format.
## New dependency Git item
You can now add a `git` block to the dependencies list:
```yaml
name: HelloWorld
main: HelloWorld.juvix
dependencies:
- .juvix-build/stdlib
- git:
url: https://my.git.repo
name: myGitRepo
ref: main
version: 0.1.0
```
Git block required fields:
* `url`: The URL of the git repository
* `ref`: The git reference that should be checked out
* `name`: The name for the dependency. This is used to name the
directory of the clone, it is required. Perhaps we could come up with a
way to automatically name the clone directory. Current ideas are to
somehow encode the URL / ref combination or use a UUID. However there's
some value in having the clone directory named in a friendly way.
NB:
* The values of the `name` fields must be unique among the git blocks in
the dependencies list.
## Behaviour
When dependencies for a package are registered, at the beginning of the
compiler pipeline, all remote dependencies are processed:
1. If it doesn't already exist, the remote dependency is cloned to
`.juvix-build/deps/$name`
2. `git fetch` is run in the clone
3. `git checkout` at the specified `ref` is run in the clone
The clone is then processed by the PathResolver in the same way as path
dependencies.
NB:
* Remote dependencies of transitive dependencies are also processed.
* The `git fetch` step is required for the case where the remote is
updated. In this case we want the user to be able to update the `ref`
field.
## Errors
1. Missing fields in the Git dependency block are YAML parse errors
2. Duplicate `name` values in the dependencies list is an error thrown
when the package file is processed
3. The `ref` doesn't exist in the clone or the clone directory is
otherwise corrupt. An error with a suggestion to `juvix clean` is given.
The package file path is used as the location in the error message.
4. Other `git` command errors (command not found, etc.), a more verbose
error is given with the arguments that were passed to the git command.
## Future work
1. Add an offline mode
2. Add a lock file mechanism that resolves branch/tag git refs to commit
hashes
* closes https://github.com/anoma/juvix/issues/2083
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
- Closes#2293.
- Closes#2319
I've added an effect for termination. It keeps track of which functions
failed the termination checker, which is run just after translating to
Internal. During typechecking, non-terminating functions are not
normalized. After typechecking, if there is at least one function which
failed the termination checker, an error is reported.
Additionally, we now properly check for termination of functions defined
in a let expression in the repl.
- Closes#2188.
This pr introduces a new syntactical statement for defining aliases:
```
syntax alias newName := oldName;
```
where `oldName` can be any name in the expression namespace. Fixity and
module aliases are not supported at the moment.
- The `newName` does not inherit the fixity of `oldName`. We have agreed
that the goal is to inherit the fixity of `oldName` except if `newName`
has a fixity statement, but this will be done in a separate pr as it
requires #2310.
- Closes#2269
Example:
```
type Sum (A B : Type) :=
| inj1 {
fst : A;
snd : B
}
| inj2 {
fst : A;
snd2 : B
};
sumSwap {A B : Type} : Sum A B -> Sum B A
| inj1@{fst; snd := y} := inj2 y fst
| inj2@{snd2 := y; fst := fst} := inj1 y fst;
```
- Closes#1642.
This pr introduces syntax for convenient record updates.
Example:
```
type Triple (A B C : Type) :=
| mkTriple {
fst : A;
snd : B;
thd : C;
};
main : Triple Nat Nat Nat;
main :=
let
p : Triple Nat Nat Nat := mkTriple 2 2 2;
p' :
Triple Nat Nat Nat :=
p @Triple{
fst := fst + 1;
snd := snd * 3
};
f : Triple Nat Nat Nat -> Triple Nat Nat Nat := (@Triple{fst := fst * 10});
in f p';
```
We write `@InductiveType{..}` to update the contents of a record. The
`@` is used for parsing. The `InductiveType` symbol indicates the type
of the record update. Inside the braces we have a list of `fieldName :=
newValue` items separated by semicolon. The `fieldName` is bound in
`newValue` with the old value of the field. Thus, we can write something
like `p @Triple{fst := fst + 1;}`.
Record updates `X@{..}` are parsed as postfix operators with higher
priority than application, so `f x y @X{q := 1}` is equivalent to `f x
(y @X{q := 1})`.
It is possible the use a record update with no argument by wrapping the
update in parentheses. See `f` in the above example.
- merge #2260 first
Allows constructors to be defined using Haskell-like Adt syntax.
E.g.
```
module Adt;
type Bool :=
| true
| false;
type Pair (A B : Type) :=
| mkPair A B;
type Nat :=
| zero
| suc Nat;
```
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
- closes#1991
This pr implements named arguments as described in #1991. It does not
yet implement optional arguments, which should be added in a later pr as
they are not required for record syntax.
# Syntax Overview
Named arguments are a convenient mehcanism to provide arguments, where
we give the arguments by name instead of by position. Anything with a
type signature can have named arguments, i.e. functions, types,
constructors and axioms.
For instance, if we have (note that named arguments can also appear on
the rhs of the `:`):
```
fun : {A B : Type} (f : A -> B) : (x : A) -> B := ... ;
```
With the traditional positional application, we would write
```
fun suc zero
```
With named arguments we can write the following:
1. `fun (f := suc) (x := zero)`.
2. We can change the order: `fun (x := zero) (f := suc)`.
3. We can group the arguments: `fun (x := zero; f := suc)`.
4. We can partially apply functions with named arguments: `fun (f :=
suc) zero`.
5. We can provide implicit arguments analogously (with braces): `fun {A
:= Nat; B := Nat} (f := suc; x := zero)`.
6. We can skip implicit arguments: `fun {B := Nat} (f := suc; x :=
zero)`.
What we cannot do:
1. Skip explicit arguments. E.g. `fun (x := zero)`.
2. Mix explicit and implicit arguments in the same group. E.g. `fun (A
:= Nat; f := suc)`
3. Provide explicit and implicit arguments in different order. E.g. `fun
(f := suc; x := zero) {A := Nat}`.
- Closes#2060
- Closes#2189
- This pr adds support for the syntax described in #2189. It does not
drop support for the old syntax.
It is possible to automatically translate juvix files to the new syntax
by using the formatter with the `--new-function-syntax` flag. E.g.
```
juvix format --in-place --new-function-syntax
```
# Syntax changes
Type signatures follow this pattern:
```
f (a1 : Expr) .. (an : Expr) : Expr
```
where each `ai` is a non-empty list of symbols. Braces are used instead
of parentheses when the argument is implicit.
Then, we have these variants:
1. Simple body. After the signature we have `:= Expr;`.
2. Clauses. The function signature is followed by a non-empty sequence
of clauses. Each clause has the form:
```
| atomPat .. atomPat := Expr
```
# Mutual recursion
Now identifiers **do not need to be defined before they are used**,
making it possible to define mutually recursive functions/types without
any special syntax.
There are some exceptions to this. We cannot forward reference a symbol
`f` in some statement `s` if between `s` and the definition of `f` there
is one of the following statements:
1. Local module
2. Import statement
3. Open statement
I think it should be possible to drop the restriction for local modules
and import statements
- Closes#2128
- Closes#2161
This pr fully implements the monadic pretty printer based on
`ExactPrint`, which respects comments. Until now, comments inside
expressions were printed after the current statement. Now they are
printed in the correct place, except when a comment occurs before
something that we don't store its location. E.g. parentheses,
semicolons, braces, colons, etc. I proposed that we irone out this issue
in a separate pr.
Since the old non-monadic algorithm is no longer necessary, I removed
it.
`AnsiText` is a type that represents some text that can be printed with
`Ansi` formatting annotations, or as plain text. It is expected that it
should have a `Semigroup` instance. This pr adds that.
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
* Closes#1992
A function identifier `fun` can be declared as an iterator with
```
syntax iterator fun;
```
For example:
```haskell
syntax iterator for;
for : {A B : Type} -> (A -> B -> A) -> A -> List B -> List A;
for f acc nil := acc;
for f acc (x :: xs) := for (f acc x) xs;
```
Iterator application syntax allows for a finite number of initializers
`acc := a` followed by a finite number of ranges `x in xs`. For example:
```
for (acc := 0) (x in lst) acc + x
```
The number of initializers plus the number of ranges must be non-zero.
An iterator application
```
fun (acc1 := a1; ..; accn := an) (x1 in b1; ..; xk in bk) body
```
gets desugared to
```
fun \{acc1 .. accn x1 .. xk := body} a1 .. an b1 .. bk
```
The `acc1`, ..., `accn`, `x1`, ..., `xk` can be patterns.
The desugaring works on a purely syntactic level. Without further
restrictions, it is not checked if the number of initializers/ranges
matches the type of the identifier. The restrictions on the number of
initializers/ranges can be specified in iterator declaration:
```
syntax iterator fun {init: n, range: k};
syntax iterator for {init: 1, range: 1};
syntax iterator map {init: 0, range: 1};
```
The attributes (`init`, `range`) in between braces are parsed as YAML to
avoid inventing and parsing a new attribute language. Both attributes
are optional.
- Closes#2089
Now the symbols inside `using {..}` and `hiding {..}` are properly
scoped, which means that they will be properly colored and will have
goto information. If the referenced module does not contain a symbol in
the list, an error will be thrown.
- Closes#2050
This pr adds the possibility to give judoc documentation in blocks
delimited by `{--` and `--}`.
- Inside these blocks, normal comments are disabled.
- It is allowed to have multiple blocks associated with the same
identifier, e.g.
```
{-- an axiom --}
{-- of type ;Type; --}
axiom a : Type;
```
- Nested blocks are *not* allowed.
- Blocks can be empty: `{-- --}`.
- The formatter respects line breaks inside blocks.
- The formatter normalizes whitespace at both ends of the block to a
single whitespace.
* Closes#1965
* Implements the `unroll` pragma to control the unrolling depth on a
per-function basis.
* Implements parsing of the `inline` pragma.
---------
Co-authored-by: janmasrovira <janmasrovira@gmail.com>
Previously we were:
* discarding the types table
* discarding the name ids state
after processing an expression in the REPL.
For example evaluating:
```
let even : _; odd : _; odd zero := false; odd (suc n) := not (even n); even zero := true; even (suc n) := not (odd n) in even 10
```
would loop in the REPL.
We noticed that the `n` in `suc n` was being given type `Type` instead
of `Nat`. This was because the name id given to n was incorrect, the
REPL started using name ids from 0 again.
We fixed this issue by storing information, including the types table
and name ids state in the Artifacts data structure that is returned when
we run the pipeline for the first time. This information is then used
when we call functions to compile / type check REPL expressions.
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
* Depends on PR #1832
* Closes#1799
* Removes Backend.C.Translation.FromInternal
* Removes `foreign` and `compile` blocks
* Removes unused test files
* Removes the old C runtime
* Removes other dead code
- Closes#1637.
A function type signature is now allowed to have a body. This is valid
for both top level and let definitions.
```
not : Bool -> Bool := λ {
| true := false
| false := true
};
```
