This PR adds support for the `String` type, String literals and string
concatenation to the Nockma backend. Support for the builtins `show` and
`intToString` is not supported.
### Example
test079.juvix
```
module test079;
import Stdlib.Prelude open;
main (s : String) : String :=
s ++str " " ++str "✨ héllo" ++str " " ++str "world ✨";
```
args.nockma
```
[quote "Juvix!"]
```
```
$ juvix compile anoma test079.juvix
$ juvix dev nockma run test079.pretty.nockma --args args.nockma
"Juvix! ✨ héllo world ✨"
```
### String representation
A String is a sequence of UTF-8 encoded bytes. We interpret these bytes
as a sequence of bits to represent the string as an integer atom in
nockma.
For example:
The string `"a"` is UTF-8 encoded as `97` which is `0b1100001` in bits.
The string `"ab"` is UTF-8 encoded at the pair of bytes: `97 98` which
is `0b1100001 0b1100010`.
When we combine the bytes into a single sequence of bits we must take
care to pad each binary representation with zeros to each byte boundary.
So the binary representation of `"ab"` as an atom is `0b110000101100010`
or `24930` as an integer atom.
### String concatenation
We use the
[cat](ea25f88cea/hoon/anoma.hoon (L215))
function in the Anoma stdlib to concatenate the bytes representing two
strings.
We need to use the block parameter `3` in the Anoma call because we want
to treat the atoms representing the strings as sequences of bytes (= 2^3
bits).
To find the relevant Nock code to call `cat` with block parameter `3` we
use the urbit dojo as follows:
```
=> anoma !=(~(cat block 3))
[8 [9 10 0 7] 9 4 10 [6 7 [0 3] 1 3] 0 2]
```
### Stdlib intercept in Evaluator
The evaluator has support for strings using `AtomHint`s, so strings can
be printed and traced. The stdlib `cat` call is also intercepted because
evaluating the unjetted hoon version is slow.
### String support in pretty nockma
In a pretty nockma file or `nock` quasi-quote you can write double
quoted string literals, e.g "abc". These are automatically translated to
UTF-8 integer atoms as in the previous section.
This PR adds support for the Anoma stdlib `sign` and `verify` APIs.
```
builtin anoma-sign
axiom anomaSign : {A : Type}
-- message to sign
-> A
-- secret key
-> Nat
-- signed message
-> Nat;
builtin anoma-verify
axiom anomaVerify : {A : Type}
-- signed message to verify
-> Nat
-- public key
-> Nat
-- message with signature removed
-> A;
```
These correspond to the
[`sign`](https://hexdocs.pm/enacl/enacl.html#sign-2) and
[`sign_open`](https://hexdocs.pm/enacl/enacl.html#sign_open-2) APIs from
libsodium respectively.
If signature verification fails in `anomaVerify`, the Anoma program
exits. We copy this behaviour in the evaluator by throwing an error in
this case.
## Notes
The Haskell Ed25519 library does not support `sign_open`. Its
verification function returns Bool, i.e it checks that the signature is
valid. The signed message is simply the concatenation of the signature
(64 bytes) and the original message so I added a function to remove the
signature from a signed message.
This PR adds support for`anomaVerifyDetached` stdlib API via a Juvix
builtin.
It has signature:
```
builtin anoma-verify-detached
axiom anomaVerifyDetached : {A : Type}
--- signature
-> Nat
--- message
-> A
--- public key
-> Nat
-> Bool;
```
The [ed25519](https://hackage.haskell.org/package/ed25519) library is
used in the evaluator becuase Anoma uses ed25519 signatures
(https://hexdocs.pm/enacl/enacl.html).
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
This PR adds support for the `anoma-decode` builtin
```
builtin anoma-decode
axiom anomaDecode : {A : Type} -> Nat -> A
```
Adds:
* An implementation of the `cue` function in Haskell
* Unit tests for `cue`
* A benchmark for `cue` applied to the Anoma / nockma stdlib
Benchmark results:
```
cue (jam stdlib): OK
36.0 ms ± 2.0 ms
```
Closes:
* https://github.com/anoma/juvix/issues/2764
This PR adds support for the `anoma-encode` builtin:
```
builtin anoma-encode
axiom anomaEncode : {A : Type} -> A -> Nat
```
In the backend this is compiled to a call to the Anoma / nockma stdlib
`jam` function.
This PR also contains:
* An implementation of the `jam` function in Haskell. This is used in
the Nockma evaluator.
* Unit tests for `jam`
* A benchmark for `jam` applied to the Anoma / nockma stdlib.
Benchmark results:
```
$ juvixbench -p 'Jam'
All
Nockma
Jam
jam stdlib: OK
109 ms ± 6.2 ms
```
# Changes
The main goal of this pr is to remove the `--target` flag for `juvix
compile` and use subcommands instead. The targets that are relevant to
normal users are found in `juvix compile --help`. Targets that are
relevant only to developers are found in `juvix dev compile --help`.
Below I list some of the changes in more detail.
## Compile targets for user-facing languages
- `juvix compile native`
- `juvix compile wasi`. I wasn't sure how to call this: `wasm`,
`wasm32-wasi`, etc. In the end I thought `wasi` was short and accurate,
but we can change it.
- `juvix compile vampir`
- `juvix compile anoma`
- `juvix compile cairo`
## *New* compile targets for internal languages
See `juvix dev compile --help`.
1. `dev compile core` has the same behaviour as `dev core
from-concrete`. The `dev core from-concrete` is redundant at the moment.
2. `dev compile tree` compiles to Tree and prints the InfoTable to the
output file wihout any additional checks.
3. `dev compile reg` compiles to Reg and prints the InfoTable to the
output file wihout any additional checks.
4. `dev compile asm` compiles to Asm and prints the InfoTable to the
output file wihout any additional checks.
5. 4. `dev compile casm` compiles to Asm and prints the Result to the
output file wihout any additional checks. TODO: should the Result be
printed or something else? At the moment the Result lacks a pretty
instance.
6.
## Optional input file
1. The input file for commands that expect a .juvix file as input is now
optional. If the argument is ommited, he main file given in the
package.yaml will be used. This applies to the following commands:
1. `juvix compile [native|wasi|geb|vampir|anoma|cairo]`
8. `juvix dev compile [core|reg|tree|casm|asm]`
1. `juvix html`
3. `juvix markdown`.
4. `juvix dev internal [typecheck|pretty]`.
5. `juvix dev [parse|scope]`
7. `juvix compile [native|wasi|geb|vampir|anoma|cairo]`
9. note that `juvix format` has not changed its behaviour.
## Refactor some C-like compiler flags
Both `juvix compile native` and `juvix compile wasi` support `--only-c`
(`-C`), `--only-preprocess` (`-E`), `--only-assemble` (`-S`). I propose
to deviate from the `gcc` style and instead use a flag with a single
argument:
- `--cstage [source|preprocess|assembly|exec(default)]`. I'm open to
suggestions. For now, I've kept the legacy flags but marked them as
deprecated in the help message.
## Remove code duplication
I've tried to reduce code duplication. This is sometimes in tension with
code readability so I've tried to find a good balance. I've tried to
make it so we don't have to jump to many different files to understand
what a single command is doing. I'm sure there is still room for
improvement.
## Other refactors
I've implemented other small refactors that I considered improved the
quality of the code.
## TODO/Future work
We should refactor commands (under `compile dev`) which still use
`module Commands.Extra.Compile` and remove it.
The purpose of this PR is to wrap the compiled main function with Nockma
code that captures the argument tuple for use when compiling `anomaGet`
calls.
* The [Anoma system
expects](c7f2d69d1e/lib/anoma/node/executor/worker.ex (L20))
to receive a function of type `ScryId -> Transaction`
* The ScryId is only used to construct the argument to the Scry
operation (i.e the anomaGet builtin in the Juvix frontend),
* When the Juvix developer writes a function to submit to Anoma they use
type `() -> Transaction`, the main function wrapper is used to capture
the ScryId argument into the subject which is then used to construct
OpScry arguments when anomaGet is compiled.
* If the Anoma system expectation changes then the wrapper code must be
changed.
We could add a transformation that checks that the main function in the
Anoma target has no arguments. However it is convenient to be able to
write functions with arguments for testing and debugging (for example
compiling directly to a logic function).
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
When we first implemented the Nockma backend we wrongly assumed that the
only entry point for Juvix compiled Nockma modules would be the main
function. Using this assumption we could add a setup step in the main
function that put the Anoma stdlib and compiled functions from the Juvix
module in a static place in the Nockma subject. References to the Anoma
stdlib and functions in the module could then be resolved statically.
However, one of the use cases for Juvix -> Nockma compilation is for
Anoma to run logic functions that are fields of a transaction. So the
user writes a Juvix program with main function that returns a
transaction. The result of the main function is passed to Anoma. When
Anoma calls the logic function on a field of the transaction, the setup
part of the main function is not run so the subject is not in the
required state. In fact, the logic function is not even callable by
Anoma because non-main functions in the Juvix module use a calling
convention that assumes the subject has a particular shape.
This PR solves the problem by making all functions in the Juvix module
use the Anoma calling convention. We make all compiled closures
(including, for example, the logic functions stored on resources in a
transaction) self contained, i.e they contain the functions library and
anoma standard library.
Modules that contain many closures produce large nockma output files
which slows down the evaluator. This will need to be fixed in the future
either with Nockma compression ([jam
serialization](https://developers.urbit.org/reference/hoon/stdlib/2p))
or otherwise. But it does not block the compilation and execution of
Anoma transactions.
Other fixes / additions:
* Extra tracing. You can now annotate output cells with a tag that will
be displayed in the output
* Unittests for listToTuple, appendRights helper functions
* Fixes for the nockma parser when parsing 'pretty nockma', specifically
stdlib calls, tags and functions_library atom.
* Adds `juvix dev nock run` command that can run a program output with
the `anoma` target.
* Remove the `nockma` target. As described above we always use the Anoma
calling convention so there's no need for a separate target for the
'juvix calling convention'
* Adds a `--profile` flag to `juvix dev nock run` which outputs a count
of Nockma ops used in the evaluation
* In tests we no longer serialise the compiled program to force full
evaluation of the compiled code. We added a negative test to check that
strings are not allowed in Nockma/Anoma programs,
it is output in a file `OUTPUT.profile` and has the following form:
```
quote : 15077
apply : 0
isCell : 0
suc : 0
= : 4517
if : 5086
seq : 5086
push : 0
call : 4896
replace : 1
hint : 8
scry : 0
trace : 0
```
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
This PR adds support for Anoma/Nockma scry OP. It is used for obtaining
values from the Anoma storage (key value store). See the [linked
issue](https://github.com/anoma/juvix/issues/2672) for details on scry.
This PR adds support for scry to the Nockma language and compilation
from the frontend via a builtin: `anoma-get`:
```
builtin anoma-get
axiom anomaGet : {Value Key : Type} -> Key -> Value
```
In the backend, the `Value` and `Key` types could be anything, they will
depend on choices of Anoma applications. The type of the returned
`Value` is unchecked. It's currently the responsibility of the user to
match the annotated type with the type of data in storage.
We will not put this builtin in the standard library. It will be exposed
in the anoma-juvix library. It's likely that the frontend `anomaGet`
function will evolve as we use it to write Anoma applications and learn
how they work.
* Closes https://github.com/anoma/juvix/issues/2672
---------
Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
See #2670 for an example which triggers the bug.
The nockma case compilation did not correctly compile case expressions
for standard (i.e not list or tuple) constructors.
Existing compilation tests (e.g Tree, Lambda Calculus) did not fail due
to the relevant `fromJust` never being evaluated due to lazy evaluation.
The tests now write out the resulting nockma file to force full
evaluation.
* Closes https://github.com/anoma/juvix/issues/2670
---------
Co-authored-by: Paul Cadman <git@paulcadman.dev>
This PR adds a `anoma` target to the `juvix compile`. This target
compiles a Juvix `main` function to a Nockma/Anoma "function". Unlike
the native, wasm, and nockma targets the main function may have any type
signature.
## Anoma calling convention
[Anoma calls
functions](6a4e15fe9c/lib/anoma/resource.ex (L122))
by evaluating the formula `[call L replace [RL args] @ S]` against a
subject equal to the function. Here `args` is a Nockma term that
evaluates to a tuple of arguments that should be passed to the function.
The anoma target compiles the `main` function to Nockma in the same way
as the nockma target. The main function is then
[wrapped](9a658465ae/src/Juvix/Compiler/Nockma/Translation/FromTree.hs (L627))
to make it compatible with the Anoma calling convention.
## Testing
The anoma calling convention is [unit
tested](9a658465ae/test/Nockma/Eval/Positive.hs (L117))
and [smoke
tested](9a658465ae/tests/smoke/Commands/compile.smoke.yaml (L159)).
This PR also adds versions of the end-to-end compilation tests. Most
tests are included, tests for builtin IO operations and string builtins
are omitted. Other tests that use Strings have been adapted to use other
types that are compatible with this backend.
## Nockma REPL
To facilitate testing the Nockma REPL can now load a nockma file as an
initial subject.
---------
Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
