git.iko.soy/effects-benchmarks
on master help sign in
Update README
3eaaf4bPatrick Thomson
README.md+100
compare with master
unifiedsplit
@@ -1 +1,101 @@
1# effects-benchmarks
1# effects-benchmarks
2+
3+This is a set of memory and time benchmarks for the various Haskell effects libraries, including `fused-effects`, `polysemy`, `freer-simple`, and `extensible-effects`. Benchmarks for equivalent `mtl` and handwritten computations are included for comparison purposes. It uses Cabal's new Backpack feature to save a great deal of boilerplate code.
4+
5+# Some results
6+
7+On a 15-inch 2017 MacBook Pro (3.1GHz Intel i7, 16GB RAM), here are results for space benchmarks:
8+
9+```
10+Countdown
11+
12+ Case Allocated
13+ fused-effects 40
14+ mtl 40
15+ polysemy 1,304
16+ freer-simple 1,096
17+ extensible-effects 1,304
18+ shallow 232
19+
20+Countdown + exc
21+
22+ Case Allocated GCs
23+ fused-effects 336
24+ mtl 336
25+ polysemy 792
26+ freer-simple 864
27+ extensible-effects 984
28+ shallow 320
29+```
30+
31+Time (`State`-based countdown):
32+
33+```
34+benchmarked Countdown/Put/fused-effects
35+time 5.692 μs (5.570 μs .. 5.835 μs)
36+
37+benchmarked Countdown/Put/mtl
38+time 5.500 μs (5.386 μs .. 5.719 μs)
39+variance introduced by outliers: 35% (moderately inflated)
40+
41+benchmarked Countdown/Put/polysemy
42+time 1.903 ms (1.845 ms .. 1.954 ms)
43+variance introduced by outliers: 72% (severely inflated)
44+
45+benchmarked Countdown/Put/freer-simple
46+time 1.324 ms (1.282 ms .. 1.361 ms)
47+
48+benchmarked Countdown/Put/extensible-effects
49+time 1.383 ms (1.343 ms .. 1.422 ms)
50+
51+benchmarked Countdown/Put/shallow
52+time 275.1 μs (270.1 μs .. 281.6 μs)
53+```
54+
55+Time (`State` + `Error` countdown):
56+
57+```
58+benchmarked Countdown/Put+Exc/fused-effects
59+time 9.829 μs (9.645 μs .. 10.04 μs)
60+
61+benchmarked Countdown/Put+Exc/mtl
62+time 9.968 μs (9.779 μs .. 10.15 μs)
63+
64+benchmarked Countdown/Put+Exc/polysemy
65+time 418.1 μs (408.9 μs .. 427.1 μs)
66+
67+benchmarked Countdown/Put+Exc/freer-simple
68+time 1.157 ms (1.120 ms .. 1.199 ms)
69+
70+benchmarked Countdown/Put+Exc/extensible-effects
71+time 1.232 ms (1.212 ms .. 1.262 ms)
72+
73+benchmarked Countdown/Put+Exc/shallow
74+time 55.12 μs (54.34 μs .. 56.29 μs)
75+```
76+
77+Time (custom HTTP DSL):
78+
79+```
80+benchmarked HTTP/fused-effects
81+time 8.311 μs (8.060 μs .. 8.592 μs)
82+
83+benchmarked HTTP/polysemy
84+time 14.99 ms (14.26 ms .. 15.80 ms)
85+
86+benchmarked HTTP/extensible-effects
87+time 1.612 ms (1.578 ms .. 1.654 ms)
88+
89+benchmarked HTTP/Deep embedding
90+time 2.027 ms (1.876 ms .. 2.184 ms)
91+
92+benchmarked HTTP/Shallow embedding
93+time 15.94 μs (15.49 μs .. 16.46 μs)
94+
95+benchmarked HTTP/freer-simple
96+time 1.478 ms (1.443 ms .. 1.515 ms)
97+```
98+
99+# Future work
100+
101+I am going to make this emit pretty graphs, or at least graphable .csv files, someday.