1
1
mirror of https://github.com/anoma/juvix.git synced 2024-12-25 16:45:20 +03:00
juvix/package.yaml

243 lines
5.6 KiB
YAML
Raw Normal View History

name: juvix
version: 0.6.8
license: GPL-3.0-only
license-file: LICENSE.md
copyright: (c) 2022- Heliax AG.
maintainer: The Compilers Team at Heliax AG <hello@heliax.dev>
author:
[
Jonathan Prieto-Cubides,
Jan Mas Rovira,
Paul Cadman,
Lukasz Czajka,
Github's contributors,
]
tested-with: ghc == 9.8.2
homepage: https://juvix.org
bug-reports: https://github.com/anoma/juvix/issues
description: The Juvix compiler
category: Compilers/Interpreters
github: anoma/juvix
2021-12-23 12:57:55 +03:00
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
flags:
# This flag can only be used in an environment that contains static libraries
# for all dependencies, including libc We use this when doing a static build
# using the ghc-musl alpine container
static:
description: Build static executable
default: false
manual: true
2021-12-23 12:57:55 +03:00
extra-source-files:
- README.md
- assets/css/*.css
- assets/js/*.js
- assets/images/*.svg
- juvix-stdlib/**/*.juvix
- include/package/**/*.juvix
- include/package-base/**/*.juvix
- runtime/c/include/**/*.h
- runtime/c/**/*.a
- runtime/rust/juvix/target/**/*.rlib
- runtime/tree/*.jvt
- runtime/vampir/*.pir
- runtime/casm/*.casm
- runtime/nockma/*.nockma
- config/config.json
- config/configure.sh
2021-12-23 12:57:55 +03:00
dependencies:
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- aeson == 2.2.*
- aeson-better-errors == 0.9.*
- aeson-pretty == 0.8.*
- ansi-terminal == 1.1.*
- array == 0.5.*
- base == 4.19.*
- base16-bytestring == 1.0.*
- base64-bytestring == 1.2.*
- bitvec == 1.1.*
- blaze-html == 0.9.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- bytestring == 0.12.*
- cereal == 0.5.*
- containers == 0.6.*
- cryptohash-sha256 == 0.11.*
- deepseq == 1.5.*
- directory == 1.3.*
- dlist == 1.0.*
- ed25519 == 0.0.*
- edit-distance == 0.2.*
Effectful Juvix tree evaluator (#2623) This pr implements two additional versions of the Juvix Tree evaluator. Now we have 1. The raw implementation that does not use effects. It throws Haskell exceptions for errors. It uses `unsafePerformIO` for traces. It relies on bang patterns to force strictness and guarantee the expected order of execution (for traces). Avoiding effects allows for improved execution efficiency. 2. [`polysemy`](https://hackage.haskell.org/package/polysemy-1.9.1.3) based implementation. 3. [`effectful-core`](https://hackage.haskell.org/package/effectful-core) based implementation. One can specify which evaluator to use thus: ``` juvix dev tree eval --evaluator XXX test.jvt ``` where XXX is one of `raw`, `polysemy`, `effectful`. # Preliminary benchmarks More thorough benchmarks should be run, but here are some preliminary results: ## Test032 (Fibonacci 20) I've adapted test032 so that it main is a single call to fibonacci of 20. Command: ``` hyperfine --warmup 2 --runs 10 'juvix dev tree eval test032.jvt --evaluator polysemy' 'juvix dev tree eval test032.jvt --evaluator raw' 'juvix dev tree eval test032.jvt --evaluator e ffectful' ``` Output: ``` Benchmark 1: juvix dev tree eval test032.jvt --evaluator polysemy Time (mean ± σ): 2.133 s ± 0.040 s [User: 2.113 s, System: 0.016 s] Range (min … max): 2.088 s … 2.227 s 10 runs Benchmark 2: juvix dev tree eval test032.jvt --evaluator raw Time (mean ± σ): 308.7 ms ± 13.8 ms [User: 293.6 ms, System: 14.1 ms] Range (min … max): 286.5 ms … 330.1 ms 10 runs Benchmark 3: juvix dev tree eval test032.jvt --evaluator effectful Time (mean ± σ): 366.0 ms ± 2.8 ms [User: 345.4 ms, System: 19.4 ms] Range (min … max): 362.5 ms … 372.6 ms 10 runs Summary juvix dev tree eval test032.jvt --evaluator raw ran 1.19 ± 0.05 times faster than juvix dev tree eval test032.jvt --evaluator effectful 6.91 ± 0.34 times faster than juvix dev tree eval test032.jvt --evaluator polysemy ``` ## Test034 (Higher-order function composition) A modified version of test034 where main defined as `call[exp](3, 12)` Command: ``` hyperfine --warmup 2 --runs 10 'juvix dev tree eval test034.jvt --evaluator polysemy' 'juvix dev tree eval test034.jvt --evaluator raw' 'juvix dev tree eval test034.jvt --evaluator effectful' ``` Output: ``` Benchmark 1: juvix dev tree eval test034.jvt --evaluator polysemy Time (mean ± σ): 7.025 s ± 0.184 s [User: 6.518 s, System: 0.469 s] Range (min … max): 6.866 s … 7.327 s 10 runs Benchmark 2: juvix dev tree eval test034.jvt --evaluator raw Time (mean ± σ): 835.6 ms ± 7.4 ms [User: 757.2 ms, System: 75.9 ms] Range (min … max): 824.7 ms … 847.4 ms 10 runs Benchmark 3: juvix dev tree eval test034.jvt --evaluator effectful Time (mean ± σ): 1.578 s ± 0.010 s [User: 1.427 s, System: 0.143 s] Range (min … max): 1.563 s … 1.595 s 10 runs Summary juvix dev tree eval test034.jvt --evaluator raw ran 1.89 ± 0.02 times faster than juvix dev tree eval test034.jvt --evaluator effectful 8.41 ± 0.23 times faster than juvix dev tree eval test034.jvt --evaluator polysemy ``` ## Test036 (Streams without memoization) A modified version of test036 where main defined as `call[nth](700, call[primes]())` Command: ``` hyperfine --warmup 2 --runs 5 'juvix dev tree eval test036.jvt --evaluator polysemy' 'juvix dev tree eval test036.jvt --evaluator raw' 'juvix dev tree eval test036.jvt --evaluator effectful' ``` Output: ``` Benchmark 1: juvix dev tree eval test036.jvt --evaluator polysemy Time (mean ± σ): 1.993 s ± 0.026 s [User: 1.946 s, System: 0.043 s] Range (min … max): 1.969 s … 2.023 s 5 runs Benchmark 2: juvix dev tree eval test036.jvt --evaluator raw Time (mean ± σ): 137.5 ms ± 7.1 ms [User: 127.5 ms, System: 8.9 ms] Range (min … max): 132.8 ms … 149.8 ms 5 runs Benchmark 3: juvix dev tree eval test036.jvt --evaluator effectful Time (mean ± σ): 329.0 ms ± 7.3 ms [User: 289.3 ms, System: 37.4 ms] Range (min … max): 319.9 ms … 336.0 ms 5 runs Summary juvix dev tree eval test036.jvt --evaluator raw ran 2.39 ± 0.13 times faster than juvix dev tree eval test036.jvt --evaluator effectful 14.50 ± 0.77 times faster than juvix dev tree eval test036.jvt --evaluator polysemy ```
2024-02-08 12:53:40 +03:00
- effectful == 2.3.*
- effectful-core == 2.3.*
- effectful-th == 1.0.*
- exceptions == 0.10.*
- extra == 1.7.*
- file-embed == 0.0.*
Run test suite in parallel (#2507) ## Overview This PR makes the compiler pipeline thread-safe so that the test suite can be run in parallel. This is achieved by: * Removing use of `{get, set, with}CurrentDir` functions. * Adding locking around shared file resources like the the global-project and internal build directory. NB: **Locking is disabled for the main compiler target**, as it is single threaded they are not required. ## Run test suite in parallel To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to your stack test arguments. For example: ``` stack test --fast --ta '+RTS -N -RTS' ``` The `-N` instructs the Haskell runtime to choose the number of threads to use based on how many processors there are on your machine. You can use `-Nn` to see the number of threads to `n`. These flags are already [set in the Makefile](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/Makefile#L26) when you or CI uses `stack test`. ## Locking The Haskell package [filelock](https://hackage.haskell.org/package/filelock) is used for locking. File locks are used instead of MVars because Juvix code does not control when new threads are created, they are created by the test suite. This means that MVars created by Juvix code will have no effect, because they are created independently on each test-suite thread. Additionally the resources we're locking live on the filesystem and so can be conveniently tagged by path. ### FileLock The filelock library is wrapped in a FileLock effect: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/Base.hs#L6-L8 There is an [IO interpreter](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/IO.hs#L8) that uses filelock and an [no-op interpreter](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/Permissive.hs#L7) that just runs actions unconditionally. ### TaggedLock To make the file locks simpler to use a TaggedLock effect is introduced: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/TaggedLock/Base.hs#L5-L11 And convenience function: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/TaggedLock.hs#L28 This allows an action to be locked, tagged by a directory that may or may not exist. For example in the following code, an action is performed on a directory `root` that may delete the directory before repopulating the files. So the lockfile cannot be stored in the `root` itself. https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Extra/Files.hs#L55-L60 ## Pipeline As noted above, we only use locking in the test suite. The main app target pipeline is single threaded and so locking is unnecessary. So the interpretation of locks is parameterised so that locking can be disabled https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Compiler/Pipeline/Run.hs#L64
2023-11-16 18:19:52 +03:00
- filelock == 0.1.*
- filepath == 1.4.*
Import tree (#2751) - Contributes to #2750 # New commands: 1. `dev import-tree scan FILE`. Scans a single file and lists all the imports in it. 2. `dev import-tree print`. Scans all files in the package and its dependencies. Builds an import dependency tree and prints it to stdin. If the `--stats` flag is given, it reports the number of scanned modules, the number of unique imports, and the length of the longest import chain. Example: this is the truncated output of `juvix dev import-tree print --stats` in the `juvix-stdlib` directory. ``` [...] Stdlib/Trait/Partial.juvix imports Stdlib/Data/String/Base.juvix Stdlib/Trait/Partial.juvix imports Stdlib/Debug/Fail.juvix Stdlib/Trait/Show.juvix imports Stdlib/Data/String/Base.juvix index.juvix imports Stdlib/Cairo/Poseidon.juvix index.juvix imports Stdlib/Data/Int/Ord.juvix index.juvix imports Stdlib/Data/Nat/Ord.juvix index.juvix imports Stdlib/Data/String/Ord.juvix index.juvix imports Stdlib/Prelude.juvix Import Tree Statistics: ======================= • Total number of modules: 56 • Total number of edges: 193 • Height (longest chain of imports): 15 ``` Bot commands support the `--scan-strategy` flag, which determines which parser we use to scan the imports. The possible values are: 1. `flatparse`. It uses the low-level [FlatParse](https://hackage.haskell.org/package/flatparse-0.5.1.0/docs/FlatParse-Basic.html) parsing library. This parser is made specifically to only parse imports and ignores the rest. So we expect this to have a much better performance. It does not have error messages. 2. `megaparsec`. It uses the normal juvix parser and we simply collect the imports from it. 4. `flatparse-megaparsec` (default). It uses the flatparse backend and fallbacks to megaparsec if it fails. # Internal changes ## Megaparsec Parser (`Concrete.FromSource`) In order to be able to run the parser during the scanning phase, I've adjusted some of the effects used in the parser: 1. I've removed the `NameIdGen` and `Files` constraints, which were unused. 2. I've removed `Reader EntryPoint`. It was used to get the `ModuleId`. Now the `ModuleId` is generated during scoping. 3. I've replaced `PathResolver` by the `TopModuleNameChecker` effect. This new effect, as the name suggests, only checks the name of the module (same rules as we had in the `PathResolver` before). It is also possible to ignore the effect, which is needed if we want to use this parser without an entrypoint. ## `PathResolver` effet refactor 1. The `WithPath` command has been removed. 2. New command `ResolvePath :: ImportScan -> PathResolver m (PackageInfo, FileExt)`. Useful for resolving imports during scanning phase. 3. New command `WithResolverRoot :: Path Abs Dir -> m a -> PathResolver m a`. Useful for switching package context. 4. New command `GetPackageInfos :: PathResolver m (HashMap (Path Abs Dir) PackageInfo)` , which returns a table with all packages. Useful to scan all dependencies. The `Package.PathResolver` has been refactored to be more like to normal `PathResolver`. We've discussed with @paulcadman the possibility to try to unify both implementations in the near future. ## Misc 1. `Package.juvix` no longer ends up in `PackageInfo.packageRelativeFiles`. 1. I've introduced string definitions for `--`, `{-` and `-}`. 2. I've fixed a bug were `.juvix.md` was detected as an invalid extension. 3. I've added `LazyHashMap` to the prelude. I've also added `ordSet` to create ordered Sets, `ordMap` for ordered maps, etc. # Benchmarks I've profiled `juvix dev import-tree --scan-strategy [megaparsec | flatparse] --stats` with optimization enabled. In the images below we see that in the megaparsec case, the scanning takes 54.8% of the total time, whereas in the flatparse case it only takes 9.6% of the total time. - **Megaparsec** ![image](https://github.com/anoma/juvix/assets/5511599/05ec42cf-d79d-4bbf-b462-c0e48593fe51) - **Flatparse** ![image](https://github.com/anoma/juvix/assets/5511599/1d7b363c-a915-463c-8dc4-613ab4b7d473) ## Hyperfine ``` hyperfine --warmup 1 'juvix dev import-tree print --scan-strategy flatparse --stats' 'juvix dev import-tree print --scan-strategy megaparsec --stats' --min-runs 20 Benchmark 1: juvix dev import-tree print --scan-strategy flatparse --stats Time (mean ± σ): 82.0 ms ± 4.5 ms [User: 64.8 ms, System: 17.3 ms] Range (min … max): 77.0 ms … 102.4 ms 37 runs Benchmark 2: juvix dev import-tree print --scan-strategy megaparsec --stats Time (mean ± σ): 174.1 ms ± 2.7 ms [User: 157.5 ms, System: 16.8 ms] Range (min … max): 169.7 ms … 181.5 ms 20 runs Summary juvix dev import-tree print --scan-strategy flatparse --stats ran 2.12 ± 0.12 times faster than juvix dev import-tree print --scan-strategy megaparsec --stats ``` In order to compare (almost) only the parsing, I've forced the scanning of each file to be performed 50 times (so that the cost of other parts get swallowed). Here are the results: ``` hyperfine --warmup 1 'juvix dev import-tree print --scan-strategy flatparse --stats' 'juvix dev import-tree print --scan-strategy megaparsec --stats' --min-runs 10 Benchmark 1: juvix dev import-tree print --scan-strategy flatparse --stats Time (mean ± σ): 189.5 ms ± 3.6 ms [User: 161.7 ms, System: 27.6 ms] Range (min … max): 185.1 ms … 197.1 ms 15 runs Benchmark 2: juvix dev import-tree print --scan-strategy megaparsec --stats Time (mean ± σ): 5.113 s ± 0.023 s [User: 5.084 s, System: 0.035 s] Range (min … max): 5.085 s … 5.148 s 10 runs Summary juvix dev import-tree print --scan-strategy flatparse --stats ran 26.99 ± 0.52 times faster than juvix dev import-tree print --scan-strategy megaparsec --stats ```
2024-05-14 11:53:33 +03:00
- flatparse == 0.5.*
- ghc == 9.8.2
- githash == 0.1.*
- hashable == 1.4.*
- language-c == 0.9.*
- libyaml == 0.1.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- megaparsec == 9.6.*
- commonmark == 0.2.*
- parsec == 3.1.*
- lens == 5.2.*
- parser-combinators == 1.3.*
- path == 0.9.*
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
- path-io == 1.8.*
- pretty == 1.1.*
- prettyprinter == 1.7.*
- prettyprinter-ansi-terminal == 1.1.*
- primitive == 0.9.*
- process == 1.6.*
- random == 1.2.*
- safe == 0.3.*
- scientific == 0.3.*
- singletons == 3.0.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- singletons-base == 3.3.*
- singletons-th == 3.3.*
- splitmix == 0.1.*
Parallel pipeline (#2779) This pr introduces parallelism in the pipeline to gain performance. I've included benchmarks at the end. - Closes #2750. # Flags: There are two new global flags: 1. `-N / --threads`. It is used to set the number of capabilities. According to [GHC documentation](https://hackage.haskell.org/package/base-4.20.0.0/docs/GHC-Conc.html#v:setNumCapabilities): _Set the number of Haskell threads that can run truly simultaneously (on separate physical processors) at any given time_. When compiling in parallel, we create this many worker threads. The default value is `-N auto`, which sets `-N` to half the number of logical cores, capped at 8. 2. `--dev-show-thread-ids`. When given, the thread id is printed in the compilation progress log. E.g. ![image](https://github.com/anoma/juvix/assets/5511599/9359fae2-0be1-43e5-8d74-faa82cba4034) # Parallel compilation 1. I've added `src/Parallel/ParallelTemplate.hs` which contains all the concurrency related code. I think it is good to keep this code separated from the actual compiler code. 2. I've added a progress log (only for the parallel driver) that outputs a log of the compilation progress, similar to what stack/cabal do. # Code changes: 1. I've removed the `setup` stage where we were registering dependencies. Instead, the dependencies are registered when the `pathResolver` is run for the first time. This way it is safer. 1. Now the `ImportTree` is needed to run the pipeline. Cycles are detected during the construction of this tree, so I've removed `Reader ImportParents` from the pipeline. 3. For the package pathresolver, we do not support parallelism yet (we could add support for it in the future, but the gains will be small). 4. When `-N1`, the pipeline remains unchanged, so performance should be the same as in the main branch (except there is a small performance degradation due to adding the `-threaded` flag). 5. I've introduced `PipelineOptions`, which are options that are used to pass options to the effects in the pipeline. 6. `PathResolver` constraint has been removed from the `upTo*` functions in the pipeline due to being redundant. 7. I've added a lot of `NFData` instances. They are needed to force the full evaluation of `Stored.ModuleInfo` in each of the threads. 2. The `Cache` effect uses [`SharedState`](https://hackage.haskell.org/package/effectful-core-2.3.0.1/docs/Effectful-State-Static-Shared.html) as opposed to [`LocalState`](https://hackage.haskell.org/package/effectful-core-2.3.0.1/docs/Effectful-Writer-Static-Local.html). Perhaps we should provide different versions. 3. I've added a `Cache` handler that accepts a setup function. The setup is triggered when a miss is detected. It is used to lazily compile the modules in parallel. # Tests 1. I've adapted the smoke test suite to ignore the progress log in the stderr. 5. I've had to adapt `tests/positive/Internal/Lambda.juvix`. Due to laziness, a crash happening in this file was not being caught. The problem is that in this file we have a lambda function with different number of patterns in their clauses, which we currently do not support (https://github.com/anoma/juvix/issues/1706). 6. I've had to comment out the definition ``` x : Box ((A : Type) → A → A) := box λ {A a := a}; ``` From the test as it was causing a crash (https://github.com/anoma/juvix/issues/2247). # Future Work 1. It should be investigated how much performance we lose by fully evaluating the `Stored.ModuleInfo`, since some information in it will be discarded. It may be possible to be more fine-grained when forcing evaluation. 8. The scanning of imports to build the import tree is sequential. Now, we build the import tree from the entry point module and only the modules that are imported from it are in the tree. However, we have discussed that at some point we should make a distinction between `juvix` _the compiler_ and `juvix` _the build tool_. When using `juvix` as a build tool it makes sense to typecheck/compile (to stored core) all modules in the project. When/if we do this, scanning imports in all modules in parallel becomes trivial. 9. The implementation of the `ParallelTemplate` uses low level primitives such as [forkIO](https://hackage.haskell.org/package/base-4.20.0.0/docs/Control-Concurrent.html#v:forkIO). At some point it should be refactored to use safer functions from the [`Effectful.Concurrent.Async`](https://hackage.haskell.org/package/effectful-2.3.0.0/docs/Effectful-Concurrent-Async.html) module. 10. The number of cores and worker threads that we spawn is determined by the command line. Ideally, we could use to import tree to compute an upper bound to the ideal number of cores to use. 11. We could add an animation that displays which modules are being compiled in parallel and which have finished being compiled. # Benchmarks On some benchmarks, I include the GHC runtime option [`-A`](https://downloads.haskell.org/ghc/latest/docs/users_guide/runtime_control.html#rts-flag--A%20%E2%9F%A8size%E2%9F%A9), which sometimes makes a good impact on performance. Thanks to @paulcadman for pointing this out. I've figured a good combination of `-N` and `-A` through trial and error (but this oviously depends on the cpu and juvix projects). ## Typecheck the standard library ### Clean run (88% faster than main): ``` hyperfine --warmup 1 --prepare 'juvix clean' 'juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432' 'juvix -N 4 typecheck Stdlib/Prelude.juvix' 'juvix-main typecheck Stdlib/Prelude.juvix' Benchmark 1: juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 Time (mean ± σ): 444.1 ms ± 6.5 ms [User: 1018.0 ms, System: 77.7 ms] Range (min … max): 432.6 ms … 455.9 ms 10 runs Benchmark 2: juvix -N 4 typecheck Stdlib/Prelude.juvix Time (mean ± σ): 628.3 ms ± 23.9 ms [User: 1227.6 ms, System: 69.5 ms] Range (min … max): 584.7 ms … 670.6 ms 10 runs Benchmark 3: juvix-main typecheck Stdlib/Prelude.juvix Time (mean ± σ): 835.9 ms ± 12.3 ms [User: 788.5 ms, System: 31.9 ms] Range (min … max): 816.0 ms … 853.6 ms 10 runs Summary juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 ran 1.41 ± 0.06 times faster than juvix -N 4 typecheck Stdlib/Prelude.juvix 1.88 ± 0.04 times faster than juvix-main typecheck Stdlib/Prelude.juvix ``` ### Cached run (43% faster than main): ``` hyperfine --warmup 1 'juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432' 'juvix -N 4 typecheck Stdlib/Prelude.juvix' 'juvix-main typecheck Stdlib/Prelude.juvix' Benchmark 1: juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 Time (mean ± σ): 241.3 ms ± 7.3 ms [User: 538.6 ms, System: 101.3 ms] Range (min … max): 231.5 ms … 251.3 ms 11 runs Benchmark 2: juvix -N 4 typecheck Stdlib/Prelude.juvix Time (mean ± σ): 235.1 ms ± 12.0 ms [User: 405.3 ms, System: 87.7 ms] Range (min … max): 216.1 ms … 253.1 ms 12 runs Benchmark 3: juvix-main typecheck Stdlib/Prelude.juvix Time (mean ± σ): 336.7 ms ± 13.3 ms [User: 269.5 ms, System: 67.1 ms] Range (min … max): 316.9 ms … 351.8 ms 10 runs Summary juvix -N 4 typecheck Stdlib/Prelude.juvix ran 1.03 ± 0.06 times faster than juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 1.43 ± 0.09 times faster than juvix-main typecheck Stdlib/Prelude.juvix ``` ## Typecheck the test suite of the containers library At the moment this is the biggest juvix project that we have. ### Clean run (105% faster than main) ``` hyperfine --warmup 1 --prepare 'juvix clean' 'juvix -N 6 typecheck Main.juvix +RTS -A67108864' 'juvix -N 4 typecheck Main.juvix' 'juvix-main typecheck Main.juvix' Benchmark 1: juvix -N 6 typecheck Main.juvix +RTS -A67108864 Time (mean ± σ): 1.006 s ± 0.011 s [User: 2.171 s, System: 0.162 s] Range (min … max): 0.991 s … 1.023 s 10 runs Benchmark 2: juvix -N 4 typecheck Main.juvix Time (mean ± σ): 1.584 s ± 0.046 s [User: 2.934 s, System: 0.149 s] Range (min … max): 1.535 s … 1.660 s 10 runs Benchmark 3: juvix-main typecheck Main.juvix Time (mean ± σ): 2.066 s ± 0.010 s [User: 1.939 s, System: 0.089 s] Range (min … max): 2.048 s … 2.077 s 10 runs Summary juvix -N 6 typecheck Main.juvix +RTS -A67108864 ran 1.57 ± 0.05 times faster than juvix -N 4 typecheck Main.juvix 2.05 ± 0.03 times faster than juvix-main typecheck Main.juvix ``` ### Cached run (54% faster than main) ``` hyperfine --warmup 1 'juvix -N 6 typecheck Main.juvix +RTS -A33554432' 'juvix -N 4 typecheck Main.juvix' 'juvix-main typecheck Main.juvix' Benchmark 1: juvix -N 6 typecheck Main.juvix +RTS -A33554432 Time (mean ± σ): 551.8 ms ± 13.2 ms [User: 1419.8 ms, System: 199.4 ms] Range (min … max): 535.2 ms … 570.6 ms 10 runs Benchmark 2: juvix -N 4 typecheck Main.juvix Time (mean ± σ): 636.7 ms ± 17.3 ms [User: 1006.3 ms, System: 196.3 ms] Range (min … max): 601.6 ms … 655.3 ms 10 runs Benchmark 3: juvix-main typecheck Main.juvix Time (mean ± σ): 847.2 ms ± 58.9 ms [User: 710.1 ms, System: 126.5 ms] Range (min … max): 731.1 ms … 890.0 ms 10 runs Summary juvix -N 6 typecheck Main.juvix +RTS -A33554432 ran 1.15 ± 0.04 times faster than juvix -N 4 typecheck Main.juvix 1.54 ± 0.11 times faster than juvix-main typecheck Main.juvix ```
2024-05-31 14:41:30 +03:00
- stm == 2.5.*
- Stream == 0.4.*
Add package lockfile support (#2388) This PR adds lock file support to the compiler pipeline. The lock file is generated whenever a compiler pipeline command (`juvix {compile, typecheck, repl}`) is run. The lock file contains all the information necessary to reproduce the whole dependency source tree. In particular for git dependencies, branch/tag references are resolved to git hash references. ## Lock file format The lock file is a YAML `juvix.lock.yaml` file written by the compiler alongside the package's `juvix.yaml` file. ``` LOCKFILE_SPEC: { dependencies: { DEPENDENCY_SPEC, dependencies: LOCKFILE_SPEC } DEPENDENCY_SPEC: PATH_SPEC | GIT_SPEC PATH_SPEC: { path: String } GIT_SPEC: { git: {url: String, ref: String, name: String } } ``` ## Example Consider a project containing the following `juvix.yaml`: ```yaml dependencies: - .juvix-build/stdlib/ - git: url: https://github.com/anoma/juvix-containers ref: v0.7.1 name: containers name: example version: 1.0.0 ``` After running `juvix compile` the following lockfile `juvix.lock.yaml` is generated. ```yaml # This file was autogenerated by Juvix version 0.5.1. # Do not edit this file manually. dependencies: - path: .juvix-build/stdlib/ dependencies: [] - git: name: containers ref: 3debbc7f5776924eb9652731b3c1982a2ee0ff24 url: https://github.com/anoma/juvix-containers dependencies: - git: name: stdlib ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45 url: https://github.com/anoma/juvix-stdlib dependencies: [] - git: name: test ref: a7ac74cac0db92e0b5e349f279d797c3788cdfdd url: https://github.com/anoma/juvix-test dependencies: - git: name: stdlib ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45 url: https://github.com/anoma/juvix-stdlib dependencies: [] ``` For subsequent runs of the juvix compile pipeline, the lock file dependency information is used. ## Behaviour when package file and lock file are out of sync If a dependency is specified in `juvix.yaml` that is not present in the lock file, an error is raised. Continuing the example above, say we add an additional dependency: ``` dependencies: - .juvix-build/stdlib/ - git: url: https://github.com/anoma/juvix-containers ref: v0.7.1 name: containers - git: url: https://github.com/anoma/juvix-test ref: v0.6.1 name: test name: example version: 1.0.0 ``` `juvix compile` will throw an error: ``` /Users/paul/tmp/lockfile/dep/juvix.yaml:1:1: error: The dependency test is declared in the package's juvix.yaml but is not declared in the lockfile: /Users/paul/tmp/lockfile/dep/juvix.lock.json Try removing /Users/paul/tmp/lockfile/dep/juvix.lock.yaml and then run Juvix again. ``` Closes: * https://github.com/anoma/juvix/issues/2334
2023-10-02 18:51:14 +03:00
- string-interpolate == 0.3.*
- template-haskell == 2.21.*
Add `juvix format` command (#1886) This PR adds `juvix format` that can be used to format either a single Juvix file or all files in a Juvix project. ## Usage ``` $ juvix format --help Usage: juvix format JUVIX_FILE_OR_PROJECT [--check] [--in-place] Format a Juvix file or Juvix project When the command is run with an unformatted file it prints the reformatted source to standard output. When the command is run with a project directory it prints a list of unformatted files in the project. Available options: JUVIX_FILE_OR_PROJECT Path to a .juvix file or to a directory containing a Juvix project. --check Do not print reformatted sources or unformatted file paths to standard output. --in-place Do not print reformatted sources to standard output. Overwrite the target's contents with the formatted version if the formatted version differs from the original content. -h,--help Show this help text ``` ## Location of main implementation The implementation is split into two components: * The src API: `format` and `formatProject` https://github.com/anoma/juvix/blob/73952ba15c90f33919c75426cf62ae7e83fd3225/src/Juvix/Formatter.hs * The CLI interface: https://github.com/anoma/juvix/blob/73952ba15c90f33919c75426cf62ae7e83fd3225/app/Commands/Format.hs ## in-place uses polysemy Resource effect The `--in-place` option makes a backup of the target file and restores it if there's an error during processing to avoid data loss. The implementation of this uses the polysemy [Resource effect](https://hackage.haskell.org/package/polysemy-1.9.0.0/docs/Polysemy-Resource.html). The recommended way to interpret the resource effect is to use `resourceToIOFinal` which makes it necessary to change the effects interpretation in main to use `Final IO`: https://github.com/anoma/juvix/blob/73952ba15c90f33919c75426cf62ae7e83fd3225/app/Main.hs#L15 ## Format input is `FilePath` The format options uses `FilePath` instead of `AppFile f` for the input file/directory used by other commands. This is because we cannot determine if the input string is a file or directory in the CLI parser (we require IO). I discussed some ideas with @janmasrovira on how to improve this in a way that would also solve other issues with CLI input file/parsing but I want to defer this to a separate PR as this one is already quite large. One consequence of Format using `FilePath` as the input option is that the code that changes the working directory to the root of the project containing the CLI input file is changed to work with `FilePath`: https://github.com/anoma/juvix/blob/f715ef6a531f63c40ac3f629dd9cfea7e867507a/app/TopCommand/Options.hs#L33 ## New dependencies This PR adds new dependencies on `temporary` and `polysemy-zoo`. `temporary` is used for `emptySystemTempFile` in the implementation of the TempFile interpreter for IO: https://github.com/anoma/juvix/blob/73952ba15c90f33919c75426cf62ae7e83fd3225/src/Juvix/Data/Effect/Files/IO.hs#L49 `polysemy-zoo` is used for the `Fresh` effect and `absorbMonadThrow` in the implementation of the pure TempFile interpreter: https://github.com/anoma/juvix/blob/73952ba15c90f33919c75426cf62ae7e83fd3225/src/Juvix/Data/Effect/Files/Pure.hs#L91 NB: The pure TempFile interpreter is not used, but it seemed a good idea to include it while it's fresh in my mind. * Closes https://github.com/anoma/juvix/issues/1777 --------- Co-authored-by: Jonathan Cubides <jonathan.cubides@uib.no>
2023-03-29 16:51:04 +03:00
- temporary == 1.3.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- text == 2.1.*
- th-utilities == 0.2.*
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
- time == 1.12.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- transformers == 0.6.*
External package dependencies (#2272) This PR adds external git dependency support to the Juvix package format. ## New dependency Git item You can now add a `git` block to the dependencies list: ```yaml name: HelloWorld main: HelloWorld.juvix dependencies: - .juvix-build/stdlib - git: url: https://my.git.repo name: myGitRepo ref: main version: 0.1.0 ``` Git block required fields: * `url`: The URL of the git repository * `ref`: The git reference that should be checked out * `name`: The name for the dependency. This is used to name the directory of the clone, it is required. Perhaps we could come up with a way to automatically name the clone directory. Current ideas are to somehow encode the URL / ref combination or use a UUID. However there's some value in having the clone directory named in a friendly way. NB: * The values of the `name` fields must be unique among the git blocks in the dependencies list. ## Behaviour When dependencies for a package are registered, at the beginning of the compiler pipeline, all remote dependencies are processed: 1. If it doesn't already exist, the remote dependency is cloned to `.juvix-build/deps/$name` 2. `git fetch` is run in the clone 3. `git checkout` at the specified `ref` is run in the clone The clone is then processed by the PathResolver in the same way as path dependencies. NB: * Remote dependencies of transitive dependencies are also processed. * The `git fetch` step is required for the case where the remote is updated. In this case we want the user to be able to update the `ref` field. ## Errors 1. Missing fields in the Git dependency block are YAML parse errors 2. Duplicate `name` values in the dependencies list is an error thrown when the package file is processed 3. The `ref` doesn't exist in the clone or the clone directory is otherwise corrupt. An error with a suggestion to `juvix clean` is given. The package file path is used as the location in the error message. 4. Other `git` command errors (command not found, etc.), a more verbose error is given with the arguments that were passed to the git command. ## Future work 1. Add an offline mode 2. Add a lock file mechanism that resolves branch/tag git refs to commit hashes * closes https://github.com/anoma/juvix/issues/2083 --------- Co-authored-by: Jan Mas Rovira <janmasrovira@gmail.com>
2023-09-01 14:37:06 +03:00
- typed-process == 0.2.*
- unicode-show == 0.1.*
- uniplate == 1.6.*
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
- unix-compat == 0.7.*
Add `juvix dev anoma {start, stop, status}` to manage an Anoma client (#3183) This PR adds the `juvix dev anoma {start, stop, status}` commands to manage a running Anoma client. The motivation for this is that we can add additional commands (e.g `indexer`, `prove`, `(mempool)-submit`) which interact with the persistent Anoma client. `juvix dev anoma start` now writes a configuration file in `<juvix_config>/anoma-client/config.yaml` which contains the host URL and port of the started Anoma client and the pid of the Anoma client process. For example: config.yaml ``` host: port: 58922 url: localhost pid: 75299 ``` The `anoma stop` command kills the Anoma client and the `anoma status` command shows the config of the currently running client. There can be at most one Anoma client running when using this mechanism. ## Dependency This PR adds a new dependency on the `unix` package. This is used for APIs to send signals to processes. ## CLI docs ### `juvix dev anoma` ``` Usage: juvix dev anoma COMMAND Subcommands related to the Anoma client Available options: -h,--help Show this help text Available commands: start Start an Anoma client status Show the status of the Anoma client stop Stop the Anoma client ``` ### `juvix dev anoma start` ``` Usage: juvix dev anoma start --anoma-dir ANOMA_DIR [-g|--foreground] [-f|--force] Start an Anoma client Available options: --anoma-dir ANOMA_DIR Path to anoma repository -g,--foreground Start the client in the foreground -f,--force Forcefully start a client, terminating any currently running client if necessary -h,--help Show this help text ``` ### `juvix dev anoma status` ``` Usage: juvix dev anoma status Show the status of the Anoma client Available options: -h,--help Show this help text ``` ### `juvix dev anoma stop` ``` Usage: juvix dev anoma stop Stop the Anoma client Available options: -h,--help Show this help text ```
2024-11-22 19:03:30 +03:00
- unix == 2.8.*
- unordered-containers == 0.2.*
- utf8-string == 1.0.*
- vector == 0.13.*
- vector-builder == 0.3.*
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
- versions == 6.0.*
- xdg-basedir == 0.2.*
- yaml == 0.11.*
2021-12-23 12:57:55 +03:00
# the tasty dependencies are here to avoid having to recompile
# juvix when running the tests.
- tasty
- tasty-hunit
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- Diff == 0.5.*
- pretty-show == 1.10.*
Improve performance of anomaEncode / anomaDecode in the Core evaluator (#2975) This PR: * Adds a new implementation of {decode, encode}ByteString functions, used by anomaEncode and anomaDecode in the Core evaluator * Adds property tests for roundtripping and benchmarks for the new functions. The old implementation used [bitvec](https://hackage.haskell.org/package/bitvec) to manipulate the ByteString. This was far too slow. The new implementation uses bit operations directly on the input integer and ByteArray. It's now possible to run [anoma-app-patterns:`Tests/Swap.juvix`](https://github.com/anoma/anoma-app-patterns/blob/feature/tests/Tests/Swap.juvix) to completion. For encoding, if the size of the output integer exceeds 64 bits (and therefore a BigInt must be used) then the new implementation has quadratic time complexity in the number of input bytes if an implementation of `ByteString -> Integer` is used as follows: ``` byteStringToIntegerLE :: ByteString -> Integer byteStringToIntegerLE = BS.foldr (\b acc -> acc `shiftL` 8 .|. fromIntegral b) 0 ``` ``` byteStringToInteger' :: ByteString -> Integer byteStringToInteger' = BS.foldl' (\acc b -> acc `shiftL` 8 .|. fromIntegral b) 0 ``` I think this is because `shiftL` is expensive for large Integers. To mitigate this I'm splitting the input ByteString into 1024 byte chunks and processing each separately. Using this we get 100x speed up at ~0.25Mb input over the non-chunked approach and linear time-complexity thereafter. ## Benchmarks The benchmarks for encoding and decoding 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 59.1 ms ± 5.3 ms decode bytes from integer: OK 338 ms ± 16 ms ``` The previous implementation would never complete for this input. Benchmarks for encoding and decoding 2 * 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 121 ms ± 8.3 ms decode bytes from integer: OK 651 ms ± 27 ms ``` Benchmarks for encoding and decoding 4 * 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 249 ms ± 17 ms decode bytes from integer: OK 1.317 s ± 16 ms ``` --------- Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
2024-08-30 20:20:18 +03:00
- hedgehog == 1.4.*
- tasty-hedgehog == 1.4.*
# benchmarks
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
- criterion == 1.6.*
- statistics == 0.16.*
- shake == 0.19.*
- colour == 2.3.*
- palette == 0.3.*
2023-01-05 19:48:26 +03:00
2021-12-23 12:57:55 +03:00
ghc-options:
# Warnings
- -Weverything
- -Wno-all-missed-specialisations
- -Wno-missed-specialisations
- -Wno-missing-export-lists
- -Wno-missing-import-lists
- -Wno-missing-kind-signatures
- -Wno-missing-safe-haskell-mode
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- -Wno-missing-role-annotations
- -Wno-missing-poly-kind-signatures
- -Wno-safe
- -Wno-unsafe
- -Wno-unused-packages
# HIE Support
- -fhide-source-paths
- -fwrite-ide-info -hiedir=.hie
# Polysemy Support
- -O2 -flate-specialise -fspecialise-aggressively
2021-12-23 12:57:55 +03:00
default-extensions:
- ApplicativeDo
- DataKinds
- DerivingStrategies
- GADTs
- GeneralizedNewtypeDeriving
- ImportQualifiedPost
- LambdaCase
- MultiWayIf
- NoFieldSelectors
- NoImplicitPrelude
- OverloadedStrings
- PatternSynonyms
Nockma compile (#2570) This PR is a snapshot of the current work on the JuvixAsm -> Nockma translation. The compilation of Juvix programs to Nockma now works so we decided to raise this PR now to avoid it getting too large. ## Juvix -> Nockma compilation You can compile a frontend Juvix file to Nockma as follows: example.juvix ``` module example; import Stdlib.Prelude open; fib : Nat → Nat → Nat → Nat | zero x1 _ := x1 | (suc n) x1 x2 := fib n x2 (x1 + x2); fibonacci (n : Nat) : Nat := fib n 0 1; sumList (xs : List Nat) : Nat := for (acc := 0) (x in xs) acc + x; main : Nat := fibonacci 9 + sumList [1; 2; 3; 4]; ``` ``` $ juvix compile -t nockma example.juvix ``` This will generate a file `example.nockma` which can be run using the nockma evaluator: ``` $ juvix dev nockma eval example.nockma ``` Alternatively you can compile JuvixAsm to Nockma: ``` $ juvix dev asm compile -t nockma example.jva ``` ## Tests We compile an evaluate the JuvixAsm tests in https://github.com/anoma/juvix/blob/cb3659e08e552ee9ca40860077e39a4070cf3303/test/Nockma/Compile/Asm/Positive.hs We currently skip some because either: 1. They are too slow to run in the current evaluator (due to arithmetic operations using the unjetted nock code from the anoma nock stdlib). 2. They trace data types like lists and booleans which are represented differently by the asm interpreter and the nock interpreter 3. They operate on raw negative numbers, nock only supports raw natural numbers ## Next steps On top of this PR we will work on improving the evaluator so that we can enable the slow compilation tests. --------- Co-authored-by: Paul Cadman <git@paulcadman.dev> Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
2024-01-17 13:15:38 +03:00
- QuasiQuotes
- RecordWildCards
- TemplateHaskell
- TypeFamilyDependencies
2021-12-23 12:57:55 +03:00
library:
source-dirs: src
2022-03-24 12:14:29 +03:00
verbatim:
default-language: GHC2021
2021-12-23 12:57:55 +03:00
executables:
Effect benchmarks (#2640) # Overview This pr implements a simple benchmark suite to compare the efficiency of [`effectful-core`](https://hackage.haskell.org/package/effectful-core) and [`polysemy`](https://hackage.haskell.org/package/polysemy). I've implemented the suite with the help of [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench). It is a simple benchmarking library that has minimal dependencies and it can be run with a default main using the same cli options as our [`tasty`](https://hackage.haskell.org/package/tasty) test suite. # How to run ``` stack run juvixbench ``` If you only want to run a particular benchmark: ``` stack run juvixbench -- -p "/Output/" ``` # Results The results show that `effectful` is the clear winner, in some cases it is extremely close to the raw version. ## State This benchmark adds the first 2 ^ 22 first naturals: ``` countRaw :: Natural -> Natural countRaw = go 0 where go :: Natural -> Natural -> Natural go acc = \case 0 -> acc m -> go (acc + m) (pred m) ``` Results: ``` State Eff State (Static): OK 25.2 ms ± 2.4 ms Sem State: OK 2.526 s ± 5.1 ms Raw State: OK 22.3 ms ± 1.5 ms ``` ## Output This benchmark collects the first 2 ^ 21 naturals in a list and adds them. ``` countdownRaw :: Natural -> Natural countdownRaw = sum' . reverse . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Eff Output (Dynamic): OK 693 ms ± 61 ms Eff Accum (Static): OK 553 ms ± 36 ms Sem Output: OK 2.606 s ± 91 ms Raw Output: OK 604 ms ± 26 ms ``` ## Reader (First Order) Repeats a constant in a list and adds it. The effects based version ask the constant value in each iteration. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (c : acc) (pred m) ``` Results: ``` Reader (First order) Eff Reader (Static): OK 103 ms ± 6.9 ms Sem Reader: OK 328 ms ± 31 ms Raw Reader: OK 106 ms ± 1.9 ms ``` ## Reader (Higher Order) Adds the first 2 ^ 21 naturals. The effects based version use `local` (from the `Reader`) effect to pass down the argument that counts the iterations. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Reader (Higher order) Eff Reader (Static): OK 720 ms ± 56 ms Sem Reader: OK 2.094 s ± 182 ms Raw Reader: OK 154 ms ± 2.2 ms ``` ## Embed IO Opens a temporary file and appends a character to it a number of times. ``` countRaw :: Natural -> IO () countRaw n = withSystemTempFile "tmp" $ \_ h -> go h n where go :: Handle -> Natural -> IO () go h = \case 0 -> return () a -> hPutChar h c >> go h (pred a) ``` Results: ``` Embed IO Raw IO: OK 464 ms ± 12 ms Eff RIO: OK 487 ms ± 3.5 ms Sem Embed IO: OK 582 ms ± 33 ms ```
2024-02-14 17:12:39 +03:00
juvixbench:
main: Main.hs
source-dirs: bench2
dependencies:
- juvix
- tasty-bench == 0.3.*
Replace `polysemy` by `effectful` (#2663) The following benchmark compares juvix 0.6.0 with polysemy and a new version (implemented in this pr) which replaces polysemy by effectful. # Typecheck standard library without caching ``` hyperfine --warmup 2 --prepare 'juvix-polysemy clean' 'juvix-polysemy typecheck Stdlib/Prelude.juvix' 'juvix-effectful typecheck Stdlib/Prelude.juvix' Benchmark 1: juvix-polysemy typecheck Stdlib/Prelude.juvix Time (mean ± σ): 3.924 s ± 0.143 s [User: 3.787 s, System: 0.084 s] Range (min … max): 3.649 s … 4.142 s 10 runs Benchmark 2: juvix-effectful typecheck Stdlib/Prelude.juvix Time (mean ± σ): 2.558 s ± 0.074 s [User: 2.430 s, System: 0.084 s] Range (min … max): 2.403 s … 2.646 s 10 runs Summary juvix-effectful typecheck Stdlib/Prelude.juvix ran 1.53 ± 0.07 times faster than juvix-polysemy typecheck Stdlib/Prelude.juvix ``` # Typecheck standard library with caching ``` hyperfine --warmup 1 'juvix-effectful typecheck Stdlib/Prelude.juvix' 'juvix-polysemy typecheck Stdlib/Prelude.juvix' --min-runs 20 Benchmark 1: juvix-effectful typecheck Stdlib/Prelude.juvix Time (mean ± σ): 1.194 s ± 0.068 s [User: 0.979 s, System: 0.211 s] Range (min … max): 1.113 s … 1.307 s 20 runs Benchmark 2: juvix-polysemy typecheck Stdlib/Prelude.juvix Time (mean ± σ): 1.237 s ± 0.083 s [User: 0.997 s, System: 0.231 s] Range (min … max): 1.061 s … 1.476 s 20 runs Summary juvix-effectful typecheck Stdlib/Prelude.juvix ran 1.04 ± 0.09 times faster than juvix-polysemy typecheck Stdlib/Prelude.juvix ```
2024-03-21 15:09:34 +03:00
- polysemy == 1.9.*
Improve performance of anomaEncode / anomaDecode in the Core evaluator (#2975) This PR: * Adds a new implementation of {decode, encode}ByteString functions, used by anomaEncode and anomaDecode in the Core evaluator * Adds property tests for roundtripping and benchmarks for the new functions. The old implementation used [bitvec](https://hackage.haskell.org/package/bitvec) to manipulate the ByteString. This was far too slow. The new implementation uses bit operations directly on the input integer and ByteArray. It's now possible to run [anoma-app-patterns:`Tests/Swap.juvix`](https://github.com/anoma/anoma-app-patterns/blob/feature/tests/Tests/Swap.juvix) to completion. For encoding, if the size of the output integer exceeds 64 bits (and therefore a BigInt must be used) then the new implementation has quadratic time complexity in the number of input bytes if an implementation of `ByteString -> Integer` is used as follows: ``` byteStringToIntegerLE :: ByteString -> Integer byteStringToIntegerLE = BS.foldr (\b acc -> acc `shiftL` 8 .|. fromIntegral b) 0 ``` ``` byteStringToInteger' :: ByteString -> Integer byteStringToInteger' = BS.foldl' (\acc b -> acc `shiftL` 8 .|. fromIntegral b) 0 ``` I think this is because `shiftL` is expensive for large Integers. To mitigate this I'm splitting the input ByteString into 1024 byte chunks and processing each separately. Using this we get 100x speed up at ~0.25Mb input over the non-chunked approach and linear time-complexity thereafter. ## Benchmarks The benchmarks for encoding and decoding 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 59.1 ms ± 5.3 ms decode bytes from integer: OK 338 ms ± 16 ms ``` The previous implementation would never complete for this input. Benchmarks for encoding and decoding 2 * 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 121 ms ± 8.3 ms decode bytes from integer: OK 651 ms ± 27 ms ``` Benchmarks for encoding and decoding 4 * 250000 bytes: ``` ByteString Encoding to/from integer encode bytes to integer: OK 249 ms ± 17 ms decode bytes from integer: OK 1.317 s ± 16 ms ``` --------- Co-authored-by: Lukasz Czajka <lukasz@heliax.dev>
2024-08-30 20:20:18 +03:00
- random
Effect benchmarks (#2640) # Overview This pr implements a simple benchmark suite to compare the efficiency of [`effectful-core`](https://hackage.haskell.org/package/effectful-core) and [`polysemy`](https://hackage.haskell.org/package/polysemy). I've implemented the suite with the help of [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench). It is a simple benchmarking library that has minimal dependencies and it can be run with a default main using the same cli options as our [`tasty`](https://hackage.haskell.org/package/tasty) test suite. # How to run ``` stack run juvixbench ``` If you only want to run a particular benchmark: ``` stack run juvixbench -- -p "/Output/" ``` # Results The results show that `effectful` is the clear winner, in some cases it is extremely close to the raw version. ## State This benchmark adds the first 2 ^ 22 first naturals: ``` countRaw :: Natural -> Natural countRaw = go 0 where go :: Natural -> Natural -> Natural go acc = \case 0 -> acc m -> go (acc + m) (pred m) ``` Results: ``` State Eff State (Static): OK 25.2 ms ± 2.4 ms Sem State: OK 2.526 s ± 5.1 ms Raw State: OK 22.3 ms ± 1.5 ms ``` ## Output This benchmark collects the first 2 ^ 21 naturals in a list and adds them. ``` countdownRaw :: Natural -> Natural countdownRaw = sum' . reverse . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Eff Output (Dynamic): OK 693 ms ± 61 ms Eff Accum (Static): OK 553 ms ± 36 ms Sem Output: OK 2.606 s ± 91 ms Raw Output: OK 604 ms ± 26 ms ``` ## Reader (First Order) Repeats a constant in a list and adds it. The effects based version ask the constant value in each iteration. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (c : acc) (pred m) ``` Results: ``` Reader (First order) Eff Reader (Static): OK 103 ms ± 6.9 ms Sem Reader: OK 328 ms ± 31 ms Raw Reader: OK 106 ms ± 1.9 ms ``` ## Reader (Higher Order) Adds the first 2 ^ 21 naturals. The effects based version use `local` (from the `Reader`) effect to pass down the argument that counts the iterations. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Reader (Higher order) Eff Reader (Static): OK 720 ms ± 56 ms Sem Reader: OK 2.094 s ± 182 ms Raw Reader: OK 154 ms ± 2.2 ms ``` ## Embed IO Opens a temporary file and appends a character to it a number of times. ``` countRaw :: Natural -> IO () countRaw n = withSystemTempFile "tmp" $ \_ h -> go h n where go :: Handle -> Natural -> IO () go h = \case 0 -> return () a -> hPutChar h c >> go h (pred a) ``` Results: ``` Embed IO Raw IO: OK 464 ms ± 12 ms Eff RIO: OK 487 ms ± 3.5 ms Sem Embed IO: OK 582 ms ± 33 ms ```
2024-02-14 17:12:39 +03:00
verbatim:
default-language: GHC2021
juvix:
2021-12-23 12:57:55 +03:00
main: Main.hs
2022-01-18 14:25:42 +03:00
source-dirs: app
2021-12-23 12:57:55 +03:00
dependencies:
- juvix
- haskeline == 0.8.*
- http-conduit == 2.3.*
upgrade to Ghc 9.8.1 (#2624) This PR updates the stackage LTS resolver to `nightly-2024-02-06` which uses GHC 9.8.1 ## Upgrade notes You will need to update your HLS to [2.6.0.0](https://github.com/haskell/haskell-language-server/releases/tag/2.6.0.0), this release contains support for GHC 9.8.1 ## Fixes ### `haskeline` / `repline` We have removed the custom haskeline / repline forks used in the build. This is because we had trouble overriding haskeline as it is bundled with GHC and the stackage resolver uses this bundled version. We were using a custom fork of haskeline to implement [mapInputT_](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/Commands/Repl.hs#L409) in the Juvix REPL, required to implement error handling. This requires private API from the Haskeline library. Instead of using a custom fork we use TemplateHaskell to obtain access to the private API we need. See [DarkArts.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/src/Juvix/Prelude/DarkArts.hs) and [HaskelineJB.hs](https://github.com/anoma/juvix/blob/15c0685c911a7ce3d2ee8c1597a1e6dd96e707bf/app/HaskelineJH.hs). To obtain access to the private API, we adapted a method from [a Tweag blogpost](https://www.tweag.io/blog/2021-01-07-haskell-dark-arts-part-i/) and [repo](https://github.com/tweag/th-jailbreak) - updating it for GHC 9.8.1. ### `aeson-better-errors` The `aeson-better-errors` library has not been updated to work with `mtl-2.3.0` so it cannot work with the new stackage resolver. We are using a [fork](https://github.com/Vekhir/aeson-better-errors.git) which has been updated. We should consider replacing this library in future, see https://github.com/anoma/juvix/issues/2621 ### `path` The `path` library now includes API `splitDrive` and `dropDrive` so we can remove our versions of those functions from the prelude. ### `with-utf8` We no longer need to depend on `with-utf8`. We were using this package for UTF-8 versions of `readFile` and `writeFile` APIs. These APIs are now available in the `text` package. ### Compiler warnings GHC 9.8.1 introduces several new compiler warnings. * We have suppressed `missing-role-annotations` and `missing-poly-kind-signatures` * We added our own versions of `head` and `tail` to work around the new `partial-tx` warning introduced for those functions in `Data.List`. * We fixed up the code to avoid the [term-variable-capture](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using-warnings.html#ghc-flag--Wterm-variable-capture) warning.
2024-02-07 12:47:48 +03:00
- mtl == 2.3.*
- optparse-applicative == 0.18.*
- repline == 0.4.*
- string-interpolate == 0.3.*
2022-03-24 12:14:29 +03:00
verbatim:
default-language: GHC2021
Parallel pipeline (#2779) This pr introduces parallelism in the pipeline to gain performance. I've included benchmarks at the end. - Closes #2750. # Flags: There are two new global flags: 1. `-N / --threads`. It is used to set the number of capabilities. According to [GHC documentation](https://hackage.haskell.org/package/base-4.20.0.0/docs/GHC-Conc.html#v:setNumCapabilities): _Set the number of Haskell threads that can run truly simultaneously (on separate physical processors) at any given time_. When compiling in parallel, we create this many worker threads. The default value is `-N auto`, which sets `-N` to half the number of logical cores, capped at 8. 2. `--dev-show-thread-ids`. When given, the thread id is printed in the compilation progress log. E.g. ![image](https://github.com/anoma/juvix/assets/5511599/9359fae2-0be1-43e5-8d74-faa82cba4034) # Parallel compilation 1. I've added `src/Parallel/ParallelTemplate.hs` which contains all the concurrency related code. I think it is good to keep this code separated from the actual compiler code. 2. I've added a progress log (only for the parallel driver) that outputs a log of the compilation progress, similar to what stack/cabal do. # Code changes: 1. I've removed the `setup` stage where we were registering dependencies. Instead, the dependencies are registered when the `pathResolver` is run for the first time. This way it is safer. 1. Now the `ImportTree` is needed to run the pipeline. Cycles are detected during the construction of this tree, so I've removed `Reader ImportParents` from the pipeline. 3. For the package pathresolver, we do not support parallelism yet (we could add support for it in the future, but the gains will be small). 4. When `-N1`, the pipeline remains unchanged, so performance should be the same as in the main branch (except there is a small performance degradation due to adding the `-threaded` flag). 5. I've introduced `PipelineOptions`, which are options that are used to pass options to the effects in the pipeline. 6. `PathResolver` constraint has been removed from the `upTo*` functions in the pipeline due to being redundant. 7. I've added a lot of `NFData` instances. They are needed to force the full evaluation of `Stored.ModuleInfo` in each of the threads. 2. The `Cache` effect uses [`SharedState`](https://hackage.haskell.org/package/effectful-core-2.3.0.1/docs/Effectful-State-Static-Shared.html) as opposed to [`LocalState`](https://hackage.haskell.org/package/effectful-core-2.3.0.1/docs/Effectful-Writer-Static-Local.html). Perhaps we should provide different versions. 3. I've added a `Cache` handler that accepts a setup function. The setup is triggered when a miss is detected. It is used to lazily compile the modules in parallel. # Tests 1. I've adapted the smoke test suite to ignore the progress log in the stderr. 5. I've had to adapt `tests/positive/Internal/Lambda.juvix`. Due to laziness, a crash happening in this file was not being caught. The problem is that in this file we have a lambda function with different number of patterns in their clauses, which we currently do not support (https://github.com/anoma/juvix/issues/1706). 6. I've had to comment out the definition ``` x : Box ((A : Type) → A → A) := box λ {A a := a}; ``` From the test as it was causing a crash (https://github.com/anoma/juvix/issues/2247). # Future Work 1. It should be investigated how much performance we lose by fully evaluating the `Stored.ModuleInfo`, since some information in it will be discarded. It may be possible to be more fine-grained when forcing evaluation. 8. The scanning of imports to build the import tree is sequential. Now, we build the import tree from the entry point module and only the modules that are imported from it are in the tree. However, we have discussed that at some point we should make a distinction between `juvix` _the compiler_ and `juvix` _the build tool_. When using `juvix` as a build tool it makes sense to typecheck/compile (to stored core) all modules in the project. When/if we do this, scanning imports in all modules in parallel becomes trivial. 9. The implementation of the `ParallelTemplate` uses low level primitives such as [forkIO](https://hackage.haskell.org/package/base-4.20.0.0/docs/Control-Concurrent.html#v:forkIO). At some point it should be refactored to use safer functions from the [`Effectful.Concurrent.Async`](https://hackage.haskell.org/package/effectful-2.3.0.0/docs/Effectful-Concurrent-Async.html) module. 10. The number of cores and worker threads that we spawn is determined by the command line. Ideally, we could use to import tree to compute an upper bound to the ideal number of cores to use. 11. We could add an animation that displays which modules are being compiled in parallel and which have finished being compiled. # Benchmarks On some benchmarks, I include the GHC runtime option [`-A`](https://downloads.haskell.org/ghc/latest/docs/users_guide/runtime_control.html#rts-flag--A%20%E2%9F%A8size%E2%9F%A9), which sometimes makes a good impact on performance. Thanks to @paulcadman for pointing this out. I've figured a good combination of `-N` and `-A` through trial and error (but this oviously depends on the cpu and juvix projects). ## Typecheck the standard library ### Clean run (88% faster than main): ``` hyperfine --warmup 1 --prepare 'juvix clean' 'juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432' 'juvix -N 4 typecheck Stdlib/Prelude.juvix' 'juvix-main typecheck Stdlib/Prelude.juvix' Benchmark 1: juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 Time (mean ± σ): 444.1 ms ± 6.5 ms [User: 1018.0 ms, System: 77.7 ms] Range (min … max): 432.6 ms … 455.9 ms 10 runs Benchmark 2: juvix -N 4 typecheck Stdlib/Prelude.juvix Time (mean ± σ): 628.3 ms ± 23.9 ms [User: 1227.6 ms, System: 69.5 ms] Range (min … max): 584.7 ms … 670.6 ms 10 runs Benchmark 3: juvix-main typecheck Stdlib/Prelude.juvix Time (mean ± σ): 835.9 ms ± 12.3 ms [User: 788.5 ms, System: 31.9 ms] Range (min … max): 816.0 ms … 853.6 ms 10 runs Summary juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 ran 1.41 ± 0.06 times faster than juvix -N 4 typecheck Stdlib/Prelude.juvix 1.88 ± 0.04 times faster than juvix-main typecheck Stdlib/Prelude.juvix ``` ### Cached run (43% faster than main): ``` hyperfine --warmup 1 'juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432' 'juvix -N 4 typecheck Stdlib/Prelude.juvix' 'juvix-main typecheck Stdlib/Prelude.juvix' Benchmark 1: juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 Time (mean ± σ): 241.3 ms ± 7.3 ms [User: 538.6 ms, System: 101.3 ms] Range (min … max): 231.5 ms … 251.3 ms 11 runs Benchmark 2: juvix -N 4 typecheck Stdlib/Prelude.juvix Time (mean ± σ): 235.1 ms ± 12.0 ms [User: 405.3 ms, System: 87.7 ms] Range (min … max): 216.1 ms … 253.1 ms 12 runs Benchmark 3: juvix-main typecheck Stdlib/Prelude.juvix Time (mean ± σ): 336.7 ms ± 13.3 ms [User: 269.5 ms, System: 67.1 ms] Range (min … max): 316.9 ms … 351.8 ms 10 runs Summary juvix -N 4 typecheck Stdlib/Prelude.juvix ran 1.03 ± 0.06 times faster than juvix -N 4 typecheck Stdlib/Prelude.juvix +RTS -A33554432 1.43 ± 0.09 times faster than juvix-main typecheck Stdlib/Prelude.juvix ``` ## Typecheck the test suite of the containers library At the moment this is the biggest juvix project that we have. ### Clean run (105% faster than main) ``` hyperfine --warmup 1 --prepare 'juvix clean' 'juvix -N 6 typecheck Main.juvix +RTS -A67108864' 'juvix -N 4 typecheck Main.juvix' 'juvix-main typecheck Main.juvix' Benchmark 1: juvix -N 6 typecheck Main.juvix +RTS -A67108864 Time (mean ± σ): 1.006 s ± 0.011 s [User: 2.171 s, System: 0.162 s] Range (min … max): 0.991 s … 1.023 s 10 runs Benchmark 2: juvix -N 4 typecheck Main.juvix Time (mean ± σ): 1.584 s ± 0.046 s [User: 2.934 s, System: 0.149 s] Range (min … max): 1.535 s … 1.660 s 10 runs Benchmark 3: juvix-main typecheck Main.juvix Time (mean ± σ): 2.066 s ± 0.010 s [User: 1.939 s, System: 0.089 s] Range (min … max): 2.048 s … 2.077 s 10 runs Summary juvix -N 6 typecheck Main.juvix +RTS -A67108864 ran 1.57 ± 0.05 times faster than juvix -N 4 typecheck Main.juvix 2.05 ± 0.03 times faster than juvix-main typecheck Main.juvix ``` ### Cached run (54% faster than main) ``` hyperfine --warmup 1 'juvix -N 6 typecheck Main.juvix +RTS -A33554432' 'juvix -N 4 typecheck Main.juvix' 'juvix-main typecheck Main.juvix' Benchmark 1: juvix -N 6 typecheck Main.juvix +RTS -A33554432 Time (mean ± σ): 551.8 ms ± 13.2 ms [User: 1419.8 ms, System: 199.4 ms] Range (min … max): 535.2 ms … 570.6 ms 10 runs Benchmark 2: juvix -N 4 typecheck Main.juvix Time (mean ± σ): 636.7 ms ± 17.3 ms [User: 1006.3 ms, System: 196.3 ms] Range (min … max): 601.6 ms … 655.3 ms 10 runs Benchmark 3: juvix-main typecheck Main.juvix Time (mean ± σ): 847.2 ms ± 58.9 ms [User: 710.1 ms, System: 126.5 ms] Range (min … max): 731.1 ms … 890.0 ms 10 runs Summary juvix -N 6 typecheck Main.juvix +RTS -A33554432 ran 1.15 ± 0.04 times faster than juvix -N 4 typecheck Main.juvix 1.54 ± 0.11 times faster than juvix-main typecheck Main.juvix ```
2024-05-31 14:41:30 +03:00
ghc-options:
- -threaded
# We enable rtsopts because we've found that tweaking the -A flag can lead
# to great performance gains. However, GHC's decumentation warns that
# enabling this may cause security problems: "...can be used to write logging
# data to arbitrary files under the security context of the running
# program..."
- -rtsopts
# We set -N1 to avoid spending time in thread initialization. We manually
# set the number of cores we want to use through the juvix -N global flag.
- -with-rtsopts=-N1
Update stackage resolver to LTS 21.6 (#2275) Stack LTS 21.6 uses GHC 9.4.5, binaries for HLS are available via ghcup. Changes required: 1. Fix warnings about type level `:` and `[]` used without backticks. 2. Fix warnings about deprecation of builtin `~` - replaced with `import Data.Type.Equality ( type (~) )` in the Prelude 3. SemVer is no longer a monoid 4. `path-io` now contains the `AnyPath` instances we were defining (thanks to Jan) so they can be removed. 5. Added `aeson-better-errors-0.9.1.1` as an extra-dep. The reason it is not part of the resolver is only because it has a strict bound on base which is not compatible with ghc 9.4.5. To work around this I've set: ``` allow-newer: true allow-newer-deps: - aeson-better-errors ``` which relaxed the upper constraint bounds for `aeson-better-errors` only. When the base constraints have been updated we can remove this workaround. 6. Use stack2cabal to generate the cabal.project file and to freeze dependency versions. https://www.stackage.org/lts-21.6/cabal.config now contains the constraint `haskeline installed`, which means that the version of haskeline that is globally installed with GHC 9.4.5 will be used, see: * https://github.com/commercialhaskell/stackage/issues/7002 GHC 9.4.5 comes with haskeline 0.8.2 preinstalled but our configuration contains the source-repository-package for haskeline 0.8.2.1 (required because we're using a fork) so if you try to run` cabal build` you get a conflict. Constraints from cabal imports cannot yet be overridden so it's not possible to get rid of this conflict using the import method. So we need to use stack2cabal with an explicit freeze file instead. 7. Remove `runTempFilePure` as this is unused and depends on `Polysemy.Fresh` in `polysemy-zoo` which is not available in the resolver. It turns out that it's not possible to use the `Fresh` effect in a pure context anyway, so it was not possible to use `runTempFilePure` for its original purpose. 8. We now use https://github.com/benz0li/ghc-musl as the base container for static linux builds, this means we don't need to maintain our own Docker container for this purpose. 9. The PR for the nightly builds is ready https://github.com/anoma/juvix-nightly-builds/pull/2, it should be merged as soon as this PR is merged. Thanks to @benz0li for maintaining https://github.com/benz0li/ghc-musl and (along with @TravisCardwell) for help with building the static binary. * Closes https://github.com/anoma/juvix/issues/2166
2023-08-11 12:49:33 +03:00
when:
- condition: flag(static)
ld-options:
- -static
- -pthread
tests:
juvix-test:
main: Main.hs
source-dirs: test
dependencies:
- juvix
2022-03-24 12:14:29 +03:00
verbatim:
default-language: GHC2021
Run test suite in parallel (#2507) ## Overview This PR makes the compiler pipeline thread-safe so that the test suite can be run in parallel. This is achieved by: * Removing use of `{get, set, with}CurrentDir` functions. * Adding locking around shared file resources like the the global-project and internal build directory. NB: **Locking is disabled for the main compiler target**, as it is single threaded they are not required. ## Run test suite in parallel To run the test suite in parallel you must add `--ta '+RTS -N -RTS'` to your stack test arguments. For example: ``` stack test --fast --ta '+RTS -N -RTS' ``` The `-N` instructs the Haskell runtime to choose the number of threads to use based on how many processors there are on your machine. You can use `-Nn` to see the number of threads to `n`. These flags are already [set in the Makefile](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/Makefile#L26) when you or CI uses `stack test`. ## Locking The Haskell package [filelock](https://hackage.haskell.org/package/filelock) is used for locking. File locks are used instead of MVars because Juvix code does not control when new threads are created, they are created by the test suite. This means that MVars created by Juvix code will have no effect, because they are created independently on each test-suite thread. Additionally the resources we're locking live on the filesystem and so can be conveniently tagged by path. ### FileLock The filelock library is wrapped in a FileLock effect: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/Base.hs#L6-L8 There is an [IO interpreter](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/IO.hs#L8) that uses filelock and an [no-op interpreter](https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/FileLock/Permissive.hs#L7) that just runs actions unconditionally. ### TaggedLock To make the file locks simpler to use a TaggedLock effect is introduced: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/TaggedLock/Base.hs#L5-L11 And convenience function: https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Data/Effect/TaggedLock.hs#L28 This allows an action to be locked, tagged by a directory that may or may not exist. For example in the following code, an action is performed on a directory `root` that may delete the directory before repopulating the files. So the lockfile cannot be stored in the `root` itself. https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Extra/Files.hs#L55-L60 ## Pipeline As noted above, we only use locking in the test suite. The main app target pipeline is single threaded and so locking is unnecessary. So the interpretation of locks is parameterised so that locking can be disabled https://github.com/anoma/juvix/blob/e6dca22cfdcff936add5e7134f9c6f20416504a5/src/Juvix/Compiler/Pipeline/Run.hs#L64
2023-11-16 18:19:52 +03:00
ghc-options:
- -threaded
2023-01-05 19:48:26 +03:00
benchmarks:
juvix-bench:
main: Main.hs
source-dirs: bench
2023-01-05 19:48:26 +03:00
dependencies:
- juvix
2023-01-05 19:48:26 +03:00
verbatim:
default-language: GHC2021