* Closes#3147
When we call a function that is currently being defined (there may be
several such due to nested local definitions), we add a reflexive edge
in the call map instead of adding an edge from the most nested
definition. For example, for
```juvix
go {A B} (f : A -> B) : List A -> List B
| nil := nil
| (elem :: next) :=
let var1 := f elem;
var2 := go f next;
in var1 :: var2;
```
we add an edge from `go` to the recursive call `go f next`, instead of
adding an edge from `var2` to `go f next` as before.
This makes the above type-check.
The following still doesn't type-check, because `next'` is not a
subpattern of the clause pattern of `go`. But this is a less pressing
problem.
```juvix
go {A B} (f : A -> B) : List A -> List B
| nil := nil
| (elem :: next) :=
let var1 := f elem;
var2 (next' : List A) : List B := go f next';
in myCons var1 (var2 next);
```
This PR:
1. Fixes the compilation of the sha256 builtin anoma lib call
A sha256 hash is 32 bytes long, not 64 bytes. This number is used when
constructing the ByteArray representation (i.e `[length payload]` cell)
of the output of Anoma stdlib sha256 call. The Anoma stdlib sha256 call
just returns the atom payload.
2. Fixes the evaluation of the sha256 stdlib call
Previously we were converting the sha256 hash bytestring to Base16
format. This is convenient when displaying the ByteString hash. However
the Anoma nock interpreter outputs the raw bytes so we must change the
builtin evaluator to match this behaviour.
After these fixes we can re-enable the test084 anoma compilation test.
This PR changes how we launch the Anoma Client to avoid a bug with
linking cryptographic APIs.
libsodium cryptographic APIs like sign-detached cannot currently be
called from within the Anoma node or client binaries. Until this is
solved we must start both the node and client from the elixir REPL.
Previously we were starting the node using the REPL and the client using
the binary.
This commit changes the `start.exs` script we were using to start the
node to now start both a node and a client.
After this change we can enable Anoma compilation test `test077`.
The output of `juvix dev anoma node --anoma-dir ANOMA_DIR` is now:
```
Anoma node and client successfully started
Listening on port 51748
```
- Fixes#3030
The `ComputeTypeInfo` transformation was incorrectly assuming that the
type of the body couldn't refer to the let item. When inferring the type
of a let, we now inline the let item(s) into the type of the body.
```
NLet Let {..} ->
let bodyTy = Info.getNodeType _letBody
in subst (_letItem ^. letItemValue) bodyTy
```
This pr adds support for getting traces from the anoma node.
I've updated the test suite so that tests that were disabled because of
traces are now being run.
There are a few tests that require atention:
1. `test028`: Gives the wrong answer.
2. `test084`: Gives the wrong answer.
4. `test074`: Expected to fail because it uses scry.
5. `test086`: Expected to fail because Anoma representation of prngs is
different than the juvix representation.
> ⚠️ Based on https://github.com/anoma/juvix/pull/3142
When using AnomaTestModeFull the compilation tests are run in debug
mode, with stdlib interception and run using the raw nock (without
stdlib interception).
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
The hoon code that generates the stdlib call is:
1a8b632463/src/Juvix/Compiler/Nockma/AnomaLib.hs (L101)
i.e the random number generator is the first argument and the width is
the second argument. We have the arguments transposed in the
corresponding Juvix builtin API so the call was failing.
This PR transposes the argumetns in the stdlib call so the builtin API
and hoon generated nock code are compatible.
* Fixes a bug in calling Anoma stdlib from Nock code
* Runs the anoma compilation test with the anoma node nockma evaluator.
I've classified the tests in 4 categories:
1. `Working`. The test works as expected.
2. `Trace`. We need more work on our end to get the traces from the
anoma node and check that they match the expected result.
3. `NodeError`. The anoma node returns `failed to prove the nock
program`.
4. `Wrong`. The anoma node returns some value that does not match the
expected value.
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
Co-authored-by: Łukasz Czajka <62751+lukaszcz@users.noreply.github.com>
Co-authored-by: Paul Cadman <git@paulcadman.dev>
* Closes#2968
* Implements detection of function-like definitions, which either:
- have some arguments on the left of `:`, or
- have at least one clause.
* Only function-like definitions are recursive.
* Non-recursive definitions are not mutually recursive either, and can
be used only after their definition. This necessitates rearranging some
definitions in existing Juvix code.
* Changes the scoping of identifiers in record updates. Now field names
on the right side don't refer to the old values of the record fields but
to identifiers in scope defined outside the record update. To refer to
old values, one needs to explicitly use record projections, e.g.
```
r@Rec{x := Rec.x r}
```
This PR adds frontend support for the Anoma Random API:
The frontend builtin APIs are:
```
builtin anoma-random-generator
axiom RandomGenerator : Type;
builtin anoma-random-generator-init
axiom randomGeneratorInit : Nat -> RandomGenerator;
builtin anoma-random-generator-split
axiom randomGeneratorSplit : RandomGenerator
-> Pair RandomGenerator RandomGenerator;
builtin anoma-random-next-bytes
axiom randomNextBytes : Nat
-> RandomGenerator
-> Pair ByteArray RandomGenerator;
```
### Nockma Evaluator
The Nockma evaluator intercepts the corresponding Anoma random stdlib
calls using the
[System.Random](https://hackage.haskell.org/package/random-1.2.1.2/docs/System-Random.html)
API. The implementation uses the
[splitmix](https://hackage.haskell.org/package/splitmix-0.1.0.5/docs/System-Random-SplitMix.html)
generator directly because it has an API to destructure the generator
into a pair of integers. We can use this to serialise the generator.
* Closes https://github.com/anoma/juvix/issues/2902
# Changes
1. Adds a new command `juvix dev anoma node`. This command runs the
anoma node.
2. Adds a flag `--anoma-dir` to `juvix dev nockma run`. When given, it
must point to the anoma clone. Then, it will run the nockma code in the
anoma node and report the result (with no traces).
# Prerequisites
1. An anoma clone at some specific commit.
```
git clone git@github.com:anoma/anoma.git
cd anoma
git checkout 98e3660b91cd55f1d9424dcff9420425ae98f5f8
# build anoma
mix deps.get
mix escript.install hex protobuf
mix compile
# build the client
mix do --app anoma_client escript.build
```
2. The `mix` command (elixir).
3. The [`grpcurl`](https://github.com/fullstorydev/grpcurl) command. To
install a single binary in `~/.local/bin` you can run:
```
curl -sSL
"https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_x86_64.tar.gz"
| tar -xz -C ~/.local/bin --no-wildcards grpcurl
```
# Testing
I've not included any test. It can be tested locally like this:
```
cd juvix/tests/Anoma/Compilation/positive
juvix compile anoma test001.juvix
echo 20 > args.debug.nockma
juvix dev nockma run --anoma-dir ~/projects/anoma test001.nockma --args args.debug.nockma
This PR adds frontend builtin support for the Anoma Resource machine
functions provided in
[resource-machine.hoon](4897751366/hoon/resource-machine.hoon),
*except* for the `prove-logic` function which still needs some
discussion.
Users must now mark the Anoma `Resource` type with
`builtin-anoma-resource` and the Anoma `Action` type with
`builtin-anoma-action`. This is required because the resource machine
functions use these types.
The compiler does not check that the constructors of `Resource` and
`Action` match the RM spec. I made this decision because the Anoma types
are sill in flux and it's easier to change if correctness is delegated
to the RM library for now. We can add the constructor checks when the
Anoma RM interface is stable.
The test file
[test085.juvix](47ba3e2746/tests/Anoma/Compilation/positive/test085.juvix)
demonstrates how each builtin should be used.
### Core Evaluator
The Core evaluator does not support these builtin functions in normal
mode. When used for normalisation (e.g when used in the constant folding
pass) the Core evaluator leaves the builtin functions unchanged.
### Nock Evaluator
The Nock evaluator does not intercept the Anoma lib functions that the
builtins correspond to in the Nock backend. It executes the underlying
Nock code instead. This means that several of the functions cannot be
tested because they're either too slow (e.g commitment) or do not have
an implementation in the Nock code (e.g addDelta).
* Closes: https://github.com/anoma/juvix/issues/3084
Most changes in this PR relate to renaming of the Anoma Nock
StandardLibrary to AnomaLibrary. This is because the Anoma library now
consists of a standard library from
[anoma.hoon](a20b5e7838/hoon/anoma.hoon)
and the resource machine library
[resource-machine.hoon](a20b5e7838/hoon/anoma.hoon).
The Anoma RM functions and value references are also added. Their
integration into the compiler pipeline will happen in a separate PR.
The Anoma Library RM functions and stdlib functions are enumerated
separately. There is a separate type for Anoma Library values because
these are compiled differently than functions.
Part of:
* https://github.com/anoma/juvix/issues/3084
This PR adds frontend support for Anoma stdlib sha256 function (aka
`shax` in Nock).
* Closes https://github.com/anoma/juvix/issues/2901
The new builtin can be declared as follows:
```
builtin anoma-sha256
axiom anomaSha256 : Nat -> ByteArray;
```
The intention is that it wraps a call to anomaEncode as follows:
```
sha256 {A} (a : A) : ByteArray := anomaSha256 (anomaEncode a);
```
### Fix for atom to ByteString
This PR also includes a commit
6205dc9ff9
to fix an issue with functions like `integerToByteArray` when called
with negative integers (the solution is to change the argument types to
Natural, as this is all we need for Anoma).
* Closes#3077
* Closes#3100
* Adds a compilation-time configuration script that creates a
`config/config.json` file which is then read by the
`Makefile`/`justfile` and embedded into the Juvix binary.
This PR adds support for the Anoma node stdlib function to the nockma
backend.
https://developers.urbit.org/reference/hoon/stdlib/2n#cury
This PR also changes the arguments placeholder value when compiling
functions and closures to make it a tuple of length equal to the
function/closure arity.
To use curry, the function argument's placeholder argument tuple must
have length equal to the function airty.
For example if we are compiling a function with arity 2, the compiled
nock function should have the form:
```
[compiled-code [0 0] env]
```
* Closes#3013
* Removes all remaining problems with evaluation duplication that could
potentially lead to an exponential blow-up in the running time.
* Adds the capacity to generate saves of temporary values in the Nock
code generation backend.
* Removes the `TempHeight` transformation on JuvixTree. It is no longer
needed.
* Removes the `ComputeCaseANF` transformation on JuvixCore from the
Anoma pipeline. It is no longer necessary.
* Closes#2804
* Requires #3003
* Front-end syntax for side conditions was implemented in #2852. This PR
implements compilation of side conditions.
* Adds side-conditions to `Match` nodes in Core. Updates Core parsing,
printing and the evaluator.
* Only side-conditions without an `else` branch are allowed in Core. If
there is an `else` branch, the side conditions are translated in
`fromInternal` into nested ifs. Because with `else` the conditions are
exhaustive, there are no implications for pattern exhaustiveness
checking.
* Adjusts the "wildcard row" case in the pattern matching compilation
algorithm to take into account the side conditions.
* Closes#2962
* Depends on #2963
* In Isabelle/HOL comments cannot appear in internal syntax. All
comments inside a Juvix definition are moved outside: to before the
definition or before the earliest function clause.
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
This PR:
* Adds a new implementation of {decode, encode}ByteString functions,
used by anomaEncode and anomaDecode in the Core evaluator
* Adds property tests for roundtripping and benchmarks for the new
functions.
The old implementation used
[bitvec](https://hackage.haskell.org/package/bitvec) to manipulate the
ByteString. This was far too slow. The new implementation uses bit
operations directly on the input integer and ByteArray.
It's now possible to run
[anoma-app-patterns:`Tests/Swap.juvix`](https://github.com/anoma/anoma-app-patterns/blob/feature/tests/Tests/Swap.juvix)
to completion.
For encoding, if the size of the output integer exceeds 64 bits (and
therefore a BigInt must be used) then the new implementation has
quadratic time complexity in the number of input bytes if an
implementation of `ByteString -> Integer` is used as follows:
```
byteStringToIntegerLE :: ByteString -> Integer
byteStringToIntegerLE = BS.foldr (\b acc -> acc `shiftL` 8 .|. fromIntegral b) 0
```
```
byteStringToInteger' :: ByteString -> Integer
byteStringToInteger' = BS.foldl' (\acc b -> acc `shiftL` 8 .|. fromIntegral b) 0
```
I think this is because `shiftL` is expensive for large Integers. To
mitigate this I'm splitting the input ByteString into 1024 byte chunks
and processing each separately. Using this we get 100x speed up at
~0.25Mb input over the non-chunked approach and linear time-complexity
thereafter.
## Benchmarks
The benchmarks for encoding and decoding 250000 bytes:
```
ByteString Encoding to/from integer
encode bytes to integer: OK
59.1 ms ± 5.3 ms
decode bytes from integer: OK
338 ms ± 16 ms
```
The previous implementation would never complete for this input.
Benchmarks for encoding and decoding 2 * 250000 bytes:
```
ByteString Encoding to/from integer
encode bytes to integer: OK
121 ms ± 8.3 ms
decode bytes from integer: OK
651 ms ± 27 ms
```
Benchmarks for encoding and decoding 4 * 250000 bytes:
```
ByteString Encoding to/from integer
encode bytes to integer: OK
249 ms ± 17 ms
decode bytes from integer: OK
1.317 s ± 16 ms
```
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
The `anoma-bytearray-{to, from}-anoma-contents` are intended to be used
to convert to/from atoms representing `ByteArrays`. These builtins are
required temporarily until Anoma Node makes ByteArray representation
uniform across all of its APIs.
We represent ByteArrays in nock as a cell:
```
[size contents]
```
Where `size` is the size of the ByteArray and `contents` is an Atom
representing the bytes in LSB ordering.
The `size` is required in general because the encoding of ByteArrays to
Atoms is ambiguous. For example the ByteArrays [0x01; 0x00] and [0x01]
are represented by `1`.
Some Anoma ByteArrays like keys and signatures are represented using on
the `contents` atom because the size is constant.
Users of Anoma APIs have to strip / add size information from ByteArrays
depending on where the data is used. The new builtins provide this
facility.
These builtins are temporary because it's been agreed with Anoma
engineering team to change the Anoma APIs to make the ByteArray
representation uniform, i.e always represent ByteArrays using `[size
content]`. When this is implemented in Anoma Node we can remove these
builtins.
```
builtin anoma-bytearray-to-anoma-contents
axiom toAnomaContents : ByteArray -> Nat;
builtin anoma-bytearray-from-anoma-contents
axiom fromAnomaContents :
-- | The size of the ByteArray
Nat
-- | The contents of the ByteArray
-> Nat
-- | The resulting ByteArray
-> ByteArray;
```
This PR adds support for ByteArray in the Anoma cryptographic functions.
```
builtin anoma-sign
axiom anomaSign : {A : Type} -> A -> ByteArray -> ByteArray;
builtin anoma-verify-with-message
axiom anomaVerifyWithMessage : {A : Type} -> ByteArray -> ByteArray -> Maybe A;
builtin anoma-sign-detached
axiom anomaSignDetached : {A : Type} -> A -> ByteArray -> ByteArray;
builtin anoma-verify-detached
axiom anomaVerifyDetached : {A : Type} -> ByteArray -> A -> ByteArray -> Bool;
```
The Anoma / Hoon Stdlib function `length` needs to be exposed as a
StdlibFunction because a ByteArray stores its length and the value
returned by `anomaSign` is not a fixed length.
This PR adds support for a builtin `ByteArray` type and associated
functions for constructing a `ByteArray` from a list of bytes and a
function to query the size of the `ByteArray`. It is only available in
the Anoma backend.
In Core / Tree, ByteArray constant is stored using a Haskell ByteString.
In Anoma the ByteArray is stored as a cell where the head is the length
of the ByteArray and the tail is an integer is an integer formed by
concatenating the bytes in the array using little-endian byte ordering.
The Nock for constructing a `ByteArray` uses the `length`, `add`,
`folder` and `lsh` functions from the Anoma hoon stdlib. See the [code
comment](fa068a30e7/src/Juvix/Compiler/Nockma/StdlibFunction.hs (L37))
for more details.
Example:
```
module test082;
import Stdlib.Prelude open;
import Stdlib.Debug.Trace open;
builtin bytearray
axiom ByteArray : Type;
builtin bytearray-from-list-byte
axiom mkByteArray : List Byte -> ByteArray;
builtin bytearray-size
axiom size : ByteArray -> Nat;
bs0 : ByteArray := mkByteArray [];
bs1 : ByteArray := mkByteArray [0x0; 0x0; 0x0];
bs2 : ByteArray := mkByteArray [0x1; 0x0; 0x0; 0x0];
bs3 : ByteArray := mkByteArray [0x2; 0x1];
bs4 : ByteArray := mkByteArray [0x100];
main : ByteArray :=
trace (size bs0)
>-> trace bs0
>-> trace (size bs1)
>-> trace bs1
>-> trace (size bs2)
>-> trace bs2
>-> trace (size bs3)
>-> trace bs3
>-> trace (size bs4)
>-> bs4;
```
Output using `tests/Anoma/Compilation/positive/test082.juvix`
```
$ juvix compile anoma -g test082.juvix
$ juvix dev nockma run test082.pretty.nockma
0
[0 0]
3
[3 0]
4
[4 1]
2
[2 258]
1
[1 0]
```
This PR adds `Byte` as a builtin with builtin functions for equality,
`byte-from-nat` and `byte-to-nat`. The standard library is updated to
include this definition with instances for `FromNatural`, `Show` and
`Eq` traits.
The `FromNatural` trait means that you can assign `Byte` values using
non-negative numeric literals.
You can use byte literals in jvc files by adding the u8 suffix to a
numeric value. For example, 1u8 represents a byte literal.
Arithmetic is not supported as the intention is for this type to be used
to construct ByteArrays of data where isn't not appropriate to modify
using arithmetic operations. We may add a separate `UInt8` type in the
future which supports arithmetic.
The Byte is supported in the native, rust and Anoma backend. Byte is not
supported in the Cairo backend because `byte-from-nat` cannot be
defined.
The primitive builtin ops for `Byte` are called `OpUInt8ToInt` and
`OpUInt8FromInt`, named because these ops work on integers and in future
we may reuse these for a separate unsigned 8-bit integer type that
supports arithmetic.
Part of:
* https://github.com/anoma/juvix/issues/2865
1. When the flatparse scanner fails and we fallback to megaparsec, a
warning is issued.
2. The flatparse scanner has been fixed so it is not confused when a
name starts with `import`.
- Closes#2923
This pr fixes a bug where all fields were assigned to be explicit
arguments in the NameSignature Builder. A single line change was enough
to fix it.
```diff
- RecordStatementField RecordField {..} -> addSymbol @s Explicit Nothing _fieldName _fieldType
+ RecordStatementField RecordField {..} -> addSymbol @s (fromIsImplicitField _fieldIsImplicit) Nothing _fieldName _fieldType
```
I've also added a compilation test for instance fields.
