git.iko.soy/streamly
on master help sign in
Add bench prefix example, additional config section
3682981Harendra Kumar
benchmark/README.md+45−4
compare with master
unifiedsplit
@@ -43,8 +43,9 @@ Run all O(1) space complexity benchmarks in `Prelude.Serial` suite:
43$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space
43$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space
44```
44```
45
45
46+Anything after a `--` is passed to the benchmark executable,
46−Anything after a `--` is passed to gauge, it basically selects all benchmarks
47+it basically selects all benchmarks starting with
47−starting with `Prelude.Serial/o-1-space` prefix.
48+`Prelude.Serial/o-1-space` prefix.
48
49
49Run a specific benchmark in `Prelude.Serial` suite:
50Run a specific benchmark in `Prelude.Serial` suite:
50
51
@@ -52,7 +53,18 @@ Run a specific benchmark in `Prelude.Serial` suite:
52$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space/generation/unfoldr
53$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space/generation/unfoldr
53```
54```
54
55
56+Run a benchmark directly instead of running it through `bench.sh`:
57+
58+```
59+$ cabal run bench:Prelude.Serial -- --quick Prelude.Serial/o-1-space/generation/unfoldr
60+```
61+
62+The options after `--` are the benchmark executable options.
63+
55−## Comparing results for regression
64+## Comparing results of arbitrary runs
65+
66+Note: use `--quick` and benchmark selection if you do not intend to wait for a
67+while.
56
68
57To compare two sets of results, first run the benchmarks at the baseline
69To compare two sets of results, first run the benchmarks at the baseline
58commit:
70commit:
@@ -71,6 +83,11 @@ $ ./bench.sh --append
71Append just adds the next set of results in the same results file. You can keep
83Append just adds the next set of results in the same results file. You can keep
72appending more results and all of them will be compared with the baseline.
84appending more results and all of them will be compared with the baseline.
73
85
86+## Comparing results of two commits
87+
88+Note: use `--quick` and benchmark selection if you do not intend to wait for a
89+while.
90+
74You can use `--compare` to compare the previous commit with the head commit:
91You can use `--compare` to compare the previous commit with the head commit:
75
92
76```
93```
@@ -102,7 +119,7 @@ $ ./bench.sh --benchmarks help
102```
119```
103
120
104You will see some benchmark suites end with `_cmp`, these are comparison
121You will see some benchmark suites end with `_cmp`, these are comparison
105−groups. If you run a comparsion group benchmark, comparison of all the
122+groups. If you run a comparison group benchmark, comparison of all the
106benchmark suites in that group will be shown in the end. For example to compare
123benchmark suites in that group will be shown in the end. For example to compare
107all array benchmark suites:
124all array benchmark suites:
108
125
@@ -122,3 +139,27 @@ You can also manually edit the file to remove a set of results if you like or
122to append results from previously saved results or from some other results
139to append results from previously saved results or from some other results
123file. After editing you can run `bench.sh` with the `--no-measure` option to
140file. After editing you can run `bench.sh` with the `--no-measure` option to
124see the reports corresponding to the results.
141see the reports corresponding to the results.
142+
143+## Additional benchmark configuration
144+
145+### Stream size
146+
147+You can specify the stream size (default is 100000) to be used for
148+benchmarking:
149+
150+```
151+$ cabal run bench:Prelude.Serial -- --quick --stream-size 1000000
152+```
153+
154+### Unicode input
155+
156+In the `FileSystem.Handle` benchmark you can specify the input file as an
157+environment variable:
158+
159+```
160+$ export Benchmark_FileSystem_Handle_InputFile=./gutenberg-500.txt
161+$ cabal run FileSystem.Handle -- --quick FileSystem.Handle/o-1-space/reduce/read/S.splitOnSeq
162+```
163+
164+The automatic tests do not test unicode input, this option is useful to specify
165+a unicode text file manually.