mirror of
https://github.com/ilyakooo0/effects-benchmarks.git
synced 2024-10-03 19:27:34 +03:00
Benchmarking Haskell extensible-effect libraries.
| instances | ||
| src | ||
| .gitignore | ||
| effects-benchmarks.cabal | ||
| LICENSE | ||
| package.yaml | ||
| README.md | ||
| Setup.hs | ||
| stack.yaml | ||
effects-benchmarks
Results
Linux – Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
*** Time benchmarks ***
benchmarked Countdown/Put/fused-effects
time 13.54 ms (13.44 ms .. 13.67 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 13.58 ms (13.54 ms .. 13.69 ms)
std dev 165.6 μs (94.38 μs .. 276.3 μs)
benchmarked Countdown/Put/mtl
time 4.885 ms (4.777 ms .. 4.954 ms)
0.998 R² (0.993 R² .. 1.000 R²)
mean 4.995 ms (4.934 ms .. 5.227 ms)
std dev 333.6 μs (54.64 μs .. 696.6 μs)
variance introduced by outliers: 37% (moderately inflated)
benchmarked Countdown/Put/polysemy
time 20.82 ms (20.58 ms .. 21.10 ms)
0.999 R² (0.997 R² .. 1.000 R²)
mean 20.82 ms (20.70 ms .. 21.00 ms)
std dev 359.1 μs (243.7 μs .. 578.8 μs)
benchmarked Countdown/Put/freer-simple
time 20.18 ms (19.82 ms .. 20.56 ms)
0.998 R² (0.997 R² .. 0.999 R²)
mean 19.90 ms (19.56 ms .. 20.53 ms)
std dev 1.084 ms (524.8 μs .. 2.078 ms)
variance introduced by outliers: 22% (moderately inflated)
benchmarked Countdown/Put/extensible-effects
time 22.01 ms (21.29 ms .. 22.72 ms)
0.997 R² (0.995 R² .. 0.999 R²)
mean 20.74 ms (19.73 ms .. 21.26 ms)
std dev 1.554 ms (611.3 μs .. 2.309 ms)
variance introduced by outliers: 30% (moderately inflated)
benchmarked Countdown/Put/shallow
time 2.432 ms (2.390 ms .. 2.477 ms)
0.998 R² (0.997 R² .. 0.999 R²)
mean 2.439 ms (2.425 ms .. 2.456 ms)
std dev 47.42 μs (39.74 μs .. 62.97 μs)
benchmarked Countdown/Put/effectful - static
time 555.5 μs (542.5 μs .. 567.2 μs)
0.995 R² (0.985 R² .. 1.000 R²)
mean 548.2 μs (544.0 μs .. 561.4 μs)
std dev 19.40 μs (5.420 μs .. 41.81 μs)
variance introduced by outliers: 12% (moderately inflated)
benchmarked Countdown/Put/effectful - dynamic
time 2.729 ms (2.688 ms .. 2.777 ms)
0.998 R² (0.996 R² .. 0.999 R²)
mean 2.736 ms (2.722 ms .. 2.761 ms)
std dev 59.64 μs (42.50 μs .. 90.32 μs)
benchmarked Countdown/Put+Exc/fused-effects
time 11.89 ms (11.76 ms .. 12.06 ms)
0.999 R² (0.999 R² .. 1.000 R²)
mean 11.93 ms (11.87 ms .. 12.05 ms)
std dev 199.3 μs (102.4 μs .. 324.7 μs)
benchmarked Countdown/Put+Exc/mtl
time 5.196 ms (5.120 ms .. 5.299 ms)
0.998 R² (0.994 R² .. 0.999 R²)
mean 5.200 ms (5.169 ms .. 5.254 ms)
std dev 108.8 μs (69.46 μs .. 167.0 μs)
benchmarked Countdown/Put+Exc/polysemy
time 29.24 ms (28.82 ms .. 29.79 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 29.51 ms (29.26 ms .. 29.95 ms)
std dev 681.2 μs (452.7 μs .. 990.8 μs)
benchmarked Countdown/Put+Exc/freer-simple
time 13.32 ms (12.65 ms .. 14.12 ms)
0.984 R² (0.974 R² .. 0.993 R²)
mean 11.50 ms (11.00 ms .. 11.99 ms)
std dev 1.209 ms (1.080 ms .. 1.435 ms)
variance introduced by outliers: 48% (moderately inflated)
benchmarked Countdown/Put+Exc/extensible-effects
time 17.03 ms (15.01 ms .. 18.76 ms)
0.956 R² (0.928 R² .. 0.980 R²)
mean 13.62 ms (12.99 ms .. 14.50 ms)
std dev 1.750 ms (1.315 ms .. 2.083 ms)
variance introduced by outliers: 56% (severely inflated)
benchmarked Countdown/Put+Exc/shallow
time 580.0 μs (475.1 μs .. 675.5 μs)
0.971 R² (0.958 R² .. 1.000 R²)
mean 573.0 μs (566.0 μs .. 584.6 μs)
std dev 14.90 μs (1.088 μs .. 19.80 μs)
benchmarked Countdown/Put+Exc/effectful - static
time 588.1 μs (520.7 μs .. 679.1 μs)
0.974 R² (0.954 R² .. 0.998 R²)
mean 602.3 μs (596.0 μs .. 612.8 μs)
std dev 13.75 μs (5.999 μs .. 17.97 μs)
benchmarked Countdown/Put+Exc/effectful - dynamic
time 2.769 ms (2.747 ms .. 2.806 ms)
0.997 R² (0.992 R² .. 1.000 R²)
mean 2.796 ms (2.783 ms .. 2.832 ms)
std dev 48.48 μs (21.03 μs .. 95.12 μs)
benchmarked HTTP/fused-effects
time 1.937 ms (1.916 ms .. 1.976 ms)
0.997 R² (0.993 R² .. 1.000 R²)
mean 1.937 ms (1.928 ms .. 1.957 ms)
std dev 29.94 μs (11.47 μs .. 54.28 μs)
benchmarked HTTP/polysemy
time 34.56 ms (33.19 ms .. 36.29 ms)
0.995 R² (0.992 R² .. 1.000 R²)
mean 33.67 ms (33.40 ms .. 34.39 ms)
std dev 841.1 μs (347.0 μs .. 1.492 ms)
benchmarked HTTP/extensible-effects
time 19.92 ms (18.98 ms .. 21.52 ms)
0.983 R² (0.970 R² .. 0.991 R²)
mean 17.74 ms (16.85 ms .. 18.62 ms)
std dev 1.942 ms (1.676 ms .. 2.228 ms)
variance introduced by outliers: 48% (moderately inflated)
benchmarked HTTP/Deep embedding
time 28.14 ms (25.55 ms .. 30.55 ms)
0.980 R² (0.966 R² .. 0.993 R²)
mean 24.73 ms (22.59 ms .. 26.22 ms)
std dev 3.694 ms (2.445 ms .. 4.726 ms)
variance introduced by outliers: 61% (severely inflated)
benchmarking HTTP/Shallow embedding ... took 11.97 s, total 40147 iterations
benchmarked HTTP/Shallow embedding
time 22.38 μs (21.23 μs .. 23.27 μs)
0.994 R² (0.977 R² .. 0.999 R²)
mean 25.00 μs (24.76 μs .. 25.35 μs)
std dev 484.0 ns (352.7 ns .. 601.7 ns)
benchmarked HTTP/freer-simple
time 19.35 ms (18.52 ms .. 20.60 ms)
0.980 R² (0.959 R² .. 0.992 R²)
mean 17.17 ms (16.27 ms .. 18.04 ms)
std dev 1.945 ms (1.653 ms .. 2.326 ms)
variance introduced by outliers: 45% (moderately inflated)
benchmarking HTTP/effectful - static ... took 11.95 s, total 38223 iterations
benchmarked HTTP/effectful - static
time 25.03 μs (16.61 μs .. 35.70 μs)
0.838 R² (0.740 R² .. 0.999 R²)
mean 25.57 μs (24.80 μs .. 27.01 μs)
std dev 1.644 μs (121.0 ns .. 2.127 μs)
variance introduced by outliers: 18% (moderately inflated)
benchmarked HTTP/effectful - dynamic
time 4.058 ms (3.662 ms .. 4.753 ms)
0.918 R² (0.847 R² .. 0.997 R²)
mean 3.886 ms (3.773 ms .. 4.122 ms)
std dev 397.4 μs (161.9 μs .. 614.5 μs)
variance introduced by outliers: 50% (severely inflated)
*** Space benchmarks (these will take some time to run) ***
Countdown
Case Allocated GCs
fused-effects 2,512 0
mtl 792 0
polysemy 4,184 0
freer-simple 1,120 0
extensible-effects 1,264 0
shallow 232 0
effectful - static 4,000,488 3
effectful - dynamic 14,401,376 13
Countdown + exc
Case Allocated GCs
fused-effects 1,080 0
mtl 608 0
polysemy 4,144 0
freer-simple 576 0
extensible-effects 688 0
shallow 0 0
effectful - static 4,002,080 3
effectful - dynamic 14,403,832 13
macOS – MacBook Pro (13-inch, 2018, Four Thunderbolt 3 Ports) – 2.3 GHz Quad-Core Intel Core i5
*** Time benchmarks ***
benchmarked Countdown/Put/fused-effects
time 13.54 ms (13.44 ms .. 13.67 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 13.58 ms (13.54 ms .. 13.69 ms)
std dev 165.6 μs (94.38 μs .. 276.3 μs)
benchmarked Countdown/Put/mtl
time 4.885 ms (4.777 ms .. 4.954 ms)
0.998 R² (0.993 R² .. 1.000 R²)
mean 4.995 ms (4.934 ms .. 5.227 ms)
std dev 333.6 μs (54.64 μs .. 696.6 μs)
variance introduced by outliers: 37% (moderately inflated)
benchmarked Countdown/Put/polysemy
time 20.82 ms (20.58 ms .. 21.10 ms)
0.999 R² (0.997 R² .. 1.000 R²)
mean 20.82 ms (20.70 ms .. 21.00 ms)
std dev 359.1 μs (243.7 μs .. 578.8 μs)
benchmarked Countdown/Put/freer-simple
time 20.18 ms (19.82 ms .. 20.56 ms)
0.998 R² (0.997 R² .. 0.999 R²)
mean 19.90 ms (19.56 ms .. 20.53 ms)
std dev 1.084 ms (524.8 μs .. 2.078 ms)
variance introduced by outliers: 22% (moderately inflated)
benchmarked Countdown/Put/extensible-effects
time 22.01 ms (21.29 ms .. 22.72 ms)
0.997 R² (0.995 R² .. 0.999 R²)
mean 20.74 ms (19.73 ms .. 21.26 ms)
std dev 1.554 ms (611.3 μs .. 2.309 ms)
variance introduced by outliers: 30% (moderately inflated)
benchmarked Countdown/Put/shallow
time 2.432 ms (2.390 ms .. 2.477 ms)
0.998 R² (0.997 R² .. 0.999 R²)
mean 2.439 ms (2.425 ms .. 2.456 ms)
std dev 47.42 μs (39.74 μs .. 62.97 μs)
benchmarked Countdown/Put/effectful - static
time 555.5 μs (542.5 μs .. 567.2 μs)
0.995 R² (0.985 R² .. 1.000 R²)
mean 548.2 μs (544.0 μs .. 561.4 μs)
std dev 19.40 μs (5.420 μs .. 41.81 μs)
variance introduced by outliers: 12% (moderately inflated)
benchmarked Countdown/Put/effectful - dynamic
time 2.729 ms (2.688 ms .. 2.777 ms)
0.998 R² (0.996 R² .. 0.999 R²)
mean 2.736 ms (2.722 ms .. 2.761 ms)
std dev 59.64 μs (42.50 μs .. 90.32 μs)
benchmarked Countdown/Put+Exc/fused-effects
time 11.89 ms (11.76 ms .. 12.06 ms)
0.999 R² (0.999 R² .. 1.000 R²)
mean 11.93 ms (11.87 ms .. 12.05 ms)
std dev 199.3 μs (102.4 μs .. 324.7 μs)
benchmarked Countdown/Put+Exc/mtl
time 5.196 ms (5.120 ms .. 5.299 ms)
0.998 R² (0.994 R² .. 0.999 R²)
mean 5.200 ms (5.169 ms .. 5.254 ms)
std dev 108.8 μs (69.46 μs .. 167.0 μs)
benchmarked Countdown/Put+Exc/polysemy
time 29.24 ms (28.82 ms .. 29.79 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 29.51 ms (29.26 ms .. 29.95 ms)
std dev 681.2 μs (452.7 μs .. 990.8 μs)
benchmarked Countdown/Put+Exc/freer-simple
time 13.32 ms (12.65 ms .. 14.12 ms)
0.984 R² (0.974 R² .. 0.993 R²)
mean 11.50 ms (11.00 ms .. 11.99 ms)
std dev 1.209 ms (1.080 ms .. 1.435 ms)
variance introduced by outliers: 48% (moderately inflated)
benchmarked Countdown/Put+Exc/extensible-effects
time 17.03 ms (15.01 ms .. 18.76 ms)
0.956 R² (0.928 R² .. 0.980 R²)
mean 13.62 ms (12.99 ms .. 14.50 ms)
std dev 1.750 ms (1.315 ms .. 2.083 ms)
variance introduced by outliers: 56% (severely inflated)
benchmarked Countdown/Put+Exc/shallow
time 580.0 μs (475.1 μs .. 675.5 μs)
0.971 R² (0.958 R² .. 1.000 R²)
mean 573.0 μs (566.0 μs .. 584.6 μs)
std dev 14.90 μs (1.088 μs .. 19.80 μs)
benchmarked Countdown/Put+Exc/effectful - static
time 588.1 μs (520.7 μs .. 679.1 μs)
0.974 R² (0.954 R² .. 0.998 R²)
mean 602.3 μs (596.0 μs .. 612.8 μs)
std dev 13.75 μs (5.999 μs .. 17.97 μs)
benchmarked Countdown/Put+Exc/effectful - dynamic
time 2.769 ms (2.747 ms .. 2.806 ms)
0.997 R² (0.992 R² .. 1.000 R²)
mean 2.796 ms (2.783 ms .. 2.832 ms)
std dev 48.48 μs (21.03 μs .. 95.12 μs)
benchmarked HTTP/fused-effects
time 1.937 ms (1.916 ms .. 1.976 ms)
0.997 R² (0.993 R² .. 1.000 R²)
mean 1.937 ms (1.928 ms .. 1.957 ms)
std dev 29.94 μs (11.47 μs .. 54.28 μs)
benchmarked HTTP/polysemy
time 34.56 ms (33.19 ms .. 36.29 ms)
0.995 R² (0.992 R² .. 1.000 R²)
mean 33.67 ms (33.40 ms .. 34.39 ms)
std dev 841.1 μs (347.0 μs .. 1.492 ms)
benchmarked HTTP/extensible-effects
time 19.92 ms (18.98 ms .. 21.52 ms)
0.983 R² (0.970 R² .. 0.991 R²)
mean 17.74 ms (16.85 ms .. 18.62 ms)
std dev 1.942 ms (1.676 ms .. 2.228 ms)
variance introduced by outliers: 48% (moderately inflated)
benchmarked HTTP/Deep embedding
time 28.14 ms (25.55 ms .. 30.55 ms)
0.980 R² (0.966 R² .. 0.993 R²)
mean 24.73 ms (22.59 ms .. 26.22 ms)
std dev 3.694 ms (2.445 ms .. 4.726 ms)
variance introduced by outliers: 61% (severely inflated)
benchmarking HTTP/Shallow embedding ... took 11.97 s, total 40147 iterations
benchmarked HTTP/Shallow embedding
time 22.38 μs (21.23 μs .. 23.27 μs)
0.994 R² (0.977 R² .. 0.999 R²)
mean 25.00 μs (24.76 μs .. 25.35 μs)
std dev 484.0 ns (352.7 ns .. 601.7 ns)
benchmarked HTTP/freer-simple
time 19.35 ms (18.52 ms .. 20.60 ms)
0.980 R² (0.959 R² .. 0.992 R²)
mean 17.17 ms (16.27 ms .. 18.04 ms)
std dev 1.945 ms (1.653 ms .. 2.326 ms)
variance introduced by outliers: 45% (moderately inflated)
benchmarking HTTP/effectful - static ... took 11.95 s, total 38223 iterations
benchmarked HTTP/effectful - static
time 25.03 μs (16.61 μs .. 35.70 μs)
0.838 R² (0.740 R² .. 0.999 R²)
mean 25.57 μs (24.80 μs .. 27.01 μs)
std dev 1.644 μs (121.0 ns .. 2.127 μs)
variance introduced by outliers: 18% (moderately inflated)
benchmarked HTTP/effectful - dynamic
time 4.058 ms (3.662 ms .. 4.753 ms)
0.918 R² (0.847 R² .. 0.997 R²)
mean 3.886 ms (3.773 ms .. 4.122 ms)
std dev 397.4 μs (161.9 μs .. 614.5 μs)
variance introduced by outliers: 50% (severely inflated)
*** Space benchmarks (these will take some time to run) ***
Countdown
Case Allocated GCs
fused-effects 2,512 0
mtl 792 0
polysemy 4,184 0
freer-simple 1,120 0
extensible-effects 1,264 0
shallow 232 0
effectful - static 4,000,488 3
effectful - dynamic 14,401,376 13
Countdown + exc
Case Allocated GCs
fused-effects 1,080 0
mtl 608 0
polysemy 4,144 0
freer-simple 576 0
extensible-effects 688 0
shallow 0 0
effectful - static 4,002,080 3
effectful - dynamic 14,403,832 13