juvix/bench2/Main.hs

module Main where

import Benchmark.Effect qualified as Effect
import Benchmark.Nockma qualified as Nockma
import Juvix.Prelude
import Test.Tasty.Bench

main :: IO ()
main =
  defaultMain
    [ Effect.bm,
      Nockma.bm
    ]
Effect benchmarks (#2640) # Overview This pr implements a simple benchmark suite to compare the efficiency of [`effectful-core`](https://hackage.haskell.org/package/effectful-core) and [`polysemy`](https://hackage.haskell.org/package/polysemy). I've implemented the suite with the help of [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench). It is a simple benchmarking library that has minimal dependencies and it can be run with a default main using the same cli options as our [`tasty`](https://hackage.haskell.org/package/tasty) test suite. # How to run ``` stack run juvixbench ``` If you only want to run a particular benchmark: ``` stack run juvixbench -- -p "/Output/" ``` # Results The results show that `effectful` is the clear winner, in some cases it is extremely close to the raw version. ## State This benchmark adds the first 2 ^ 22 first naturals: ``` countRaw :: Natural -> Natural countRaw = go 0 where go :: Natural -> Natural -> Natural go acc = \case 0 -> acc m -> go (acc + m) (pred m) ``` Results: ``` State Eff State (Static): OK 25.2 ms ± 2.4 ms Sem State: OK 2.526 s ± 5.1 ms Raw State: OK 22.3 ms ± 1.5 ms ``` ## Output This benchmark collects the first 2 ^ 21 naturals in a list and adds them. ``` countdownRaw :: Natural -> Natural countdownRaw = sum' . reverse . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Eff Output (Dynamic): OK 693 ms ± 61 ms Eff Accum (Static): OK 553 ms ± 36 ms Sem Output: OK 2.606 s ± 91 ms Raw Output: OK 604 ms ± 26 ms ``` ## Reader (First Order) Repeats a constant in a list and adds it. The effects based version ask the constant value in each iteration. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (c : acc) (pred m) ``` Results: ``` Reader (First order) Eff Reader (Static): OK 103 ms ± 6.9 ms Sem Reader: OK 328 ms ± 31 ms Raw Reader: OK 106 ms ± 1.9 ms ``` ## Reader (Higher Order) Adds the first 2 ^ 21 naturals. The effects based version use `local` (from the `Reader`) effect to pass down the argument that counts the iterations. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Reader (Higher order) Eff Reader (Static): OK 720 ms ± 56 ms Sem Reader: OK 2.094 s ± 182 ms Raw Reader: OK 154 ms ± 2.2 ms ``` ## Embed IO Opens a temporary file and appends a character to it a number of times. ``` countRaw :: Natural -> IO () countRaw n = withSystemTempFile "tmp" $ \_ h -> go h n where go :: Handle -> Natural -> IO () go h = \case 0 -> return () a -> hPutChar h c >> go h (pred a) ``` Results: ``` Embed IO Raw IO: OK 464 ms ± 12 ms Eff RIO: OK 487 ms ± 3.5 ms Sem Embed IO: OK 582 ms ± 33 ms ``` 2024-02-14 17:12:39 +03:00			`module Main where`

			`import Benchmark.Effect qualified as Effect`
Add support for anoma-encode builtin (#2766) This PR adds support for the `anoma-encode` builtin: ``` builtin anoma-encode axiom anomaEncode : {A : Type} -> A -> Nat ``` In the backend this is compiled to a call to the Anoma / nockma stdlib `jam` function. This PR also contains: * An implementation of the `jam` function in Haskell. This is used in the Nockma evaluator. * Unit tests for `jam` * A benchmark for `jam` applied to the Anoma / nockma stdlib. Benchmark results: ``` $ juvixbench -p 'Jam' All Nockma Jam jam stdlib: OK 109 ms ± 6.2 ms ``` 2024-05-14 19:45:49 +03:00			`import Benchmark.Nockma qualified as Nockma`
Effect benchmarks (#2640) # Overview This pr implements a simple benchmark suite to compare the efficiency of [`effectful-core`](https://hackage.haskell.org/package/effectful-core) and [`polysemy`](https://hackage.haskell.org/package/polysemy). I've implemented the suite with the help of [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench). It is a simple benchmarking library that has minimal dependencies and it can be run with a default main using the same cli options as our [`tasty`](https://hackage.haskell.org/package/tasty) test suite. # How to run ``` stack run juvixbench ``` If you only want to run a particular benchmark: ``` stack run juvixbench -- -p "/Output/" ``` # Results The results show that `effectful` is the clear winner, in some cases it is extremely close to the raw version. ## State This benchmark adds the first 2 ^ 22 first naturals: ``` countRaw :: Natural -> Natural countRaw = go 0 where go :: Natural -> Natural -> Natural go acc = \case 0 -> acc m -> go (acc + m) (pred m) ``` Results: ``` State Eff State (Static): OK 25.2 ms ± 2.4 ms Sem State: OK 2.526 s ± 5.1 ms Raw State: OK 22.3 ms ± 1.5 ms ``` ## Output This benchmark collects the first 2 ^ 21 naturals in a list and adds them. ``` countdownRaw :: Natural -> Natural countdownRaw = sum' . reverse . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Eff Output (Dynamic): OK 693 ms ± 61 ms Eff Accum (Static): OK 553 ms ± 36 ms Sem Output: OK 2.606 s ± 91 ms Raw Output: OK 604 ms ± 26 ms ``` ## Reader (First Order) Repeats a constant in a list and adds it. The effects based version ask the constant value in each iteration. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (c : acc) (pred m) ``` Results: ``` Reader (First order) Eff Reader (Static): OK 103 ms ± 6.9 ms Sem Reader: OK 328 ms ± 31 ms Raw Reader: OK 106 ms ± 1.9 ms ``` ## Reader (Higher Order) Adds the first 2 ^ 21 naturals. The effects based version use `local` (from the `Reader`) effect to pass down the argument that counts the iterations. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Reader (Higher order) Eff Reader (Static): OK 720 ms ± 56 ms Sem Reader: OK 2.094 s ± 182 ms Raw Reader: OK 154 ms ± 2.2 ms ``` ## Embed IO Opens a temporary file and appends a character to it a number of times. ``` countRaw :: Natural -> IO () countRaw n = withSystemTempFile "tmp" $ \_ h -> go h n where go :: Handle -> Natural -> IO () go h = \case 0 -> return () a -> hPutChar h c >> go h (pred a) ``` Results: ``` Embed IO Raw IO: OK 464 ms ± 12 ms Eff RIO: OK 487 ms ± 3.5 ms Sem Embed IO: OK 582 ms ± 33 ms ``` 2024-02-14 17:12:39 +03:00			`import Juvix.Prelude`
			`import Test.Tasty.Bench`

			`main :: IO ()`
			`main =`
			`defaultMain`
Add support for anoma-encode builtin (#2766) This PR adds support for the `anoma-encode` builtin: ``` builtin anoma-encode axiom anomaEncode : {A : Type} -> A -> Nat ``` In the backend this is compiled to a call to the Anoma / nockma stdlib `jam` function. This PR also contains: * An implementation of the `jam` function in Haskell. This is used in the Nockma evaluator. * Unit tests for `jam` * A benchmark for `jam` applied to the Anoma / nockma stdlib. Benchmark results: ``` $ juvixbench -p 'Jam' All Nockma Jam jam stdlib: OK 109 ms ± 6.2 ms ``` 2024-05-14 19:45:49 +03:00			`[ Effect.bm,`
			`Nockma.bm`
Effect benchmarks (#2640) # Overview This pr implements a simple benchmark suite to compare the efficiency of [`effectful-core`](https://hackage.haskell.org/package/effectful-core) and [`polysemy`](https://hackage.haskell.org/package/polysemy). I've implemented the suite with the help of [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench). It is a simple benchmarking library that has minimal dependencies and it can be run with a default main using the same cli options as our [`tasty`](https://hackage.haskell.org/package/tasty) test suite. # How to run ``` stack run juvixbench ``` If you only want to run a particular benchmark: ``` stack run juvixbench -- -p "/Output/" ``` # Results The results show that `effectful` is the clear winner, in some cases it is extremely close to the raw version. ## State This benchmark adds the first 2 ^ 22 first naturals: ``` countRaw :: Natural -> Natural countRaw = go 0 where go :: Natural -> Natural -> Natural go acc = \case 0 -> acc m -> go (acc + m) (pred m) ``` Results: ``` State Eff State (Static): OK 25.2 ms ± 2.4 ms Sem State: OK 2.526 s ± 5.1 ms Raw State: OK 22.3 ms ± 1.5 ms ``` ## Output This benchmark collects the first 2 ^ 21 naturals in a list and adds them. ``` countdownRaw :: Natural -> Natural countdownRaw = sum' . reverse . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Eff Output (Dynamic): OK 693 ms ± 61 ms Eff Accum (Static): OK 553 ms ± 36 ms Sem Output: OK 2.606 s ± 91 ms Raw Output: OK 604 ms ± 26 ms ``` ## Reader (First Order) Repeats a constant in a list and adds it. The effects based version ask the constant value in each iteration. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (c : acc) (pred m) ``` Results: ``` Reader (First order) Eff Reader (Static): OK 103 ms ± 6.9 ms Sem Reader: OK 328 ms ± 31 ms Raw Reader: OK 106 ms ± 1.9 ms ``` ## Reader (Higher Order) Adds the first 2 ^ 21 naturals. The effects based version use `local` (from the `Reader`) effect to pass down the argument that counts the iterations. ``` countRaw :: Natural -> Natural countRaw = sum' . go [] where go :: [Natural] -> Natural -> [Natural] go acc = \case 0 -> acc m -> go (m : acc) (pred m) ``` Results: ``` Reader (Higher order) Eff Reader (Static): OK 720 ms ± 56 ms Sem Reader: OK 2.094 s ± 182 ms Raw Reader: OK 154 ms ± 2.2 ms ``` ## Embed IO Opens a temporary file and appends a character to it a number of times. ``` countRaw :: Natural -> IO () countRaw n = withSystemTempFile "tmp" $ \_ h -> go h n where go :: Handle -> Natural -> IO () go h = \case 0 -> return () a -> hPutChar h c >> go h (pred a) ``` Results: ``` Embed IO Raw IO: OK 464 ms ± 12 ms Eff RIO: OK 487 ms ± 3.5 ms Sem Embed IO: OK 582 ms ± 33 ms ``` 2024-02-14 17:12:39 +03:00			`]`