@@ -7,34 +7,52 @@ options to run it.
77
88## Quick start
99
10−Run these commands from the root of the repo.
10+You must be in the repo root directory to run these commands.
1111
12−To run the default benchmarks:
12+Run the default benchmark suites:
1313
1414```
15−$ ./bench.sh
15+$ ./bench.sh --quick
16+```
17+
18+You can remove the `--quick` option to run benchmark with lower speed but
19+better accuracy, or use `--slow` to even further lower the speed and increase
20+the accuracy a bit.
21+
22+Show available benchmark suites:
23+
24+```
25+$ ./bench.sh --benchmarks help
26+```
27+
28+Run all benchmark suites in the `serial_grp` group:
29+
30+```
31+$ ./bench.sh --benchmarks serial_grp
1632```
1733
18−To run all benchmarks:
34+Run `Prelude.Serial` and `Data.Parser` benchmark suites:
1935
2036```
21−$ ./bench.sh --benchmarks all
37+$ ./bench.sh --benchmarks "Prelude.Serial Data.Parser"
2238```
2339
24−To run `linear` and `linear-async` benchmarks:
40+Run all O(1) space complexity benchmarks in `Prelude.Serial` suite:
2541
2642```
27−$ ./bench.sh --benchmarks "linear linear-async"
43+$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space
2844```
2945
30−To run only the base benchmark and only the benchmarks prefixed with
31−`StreamD` in that (anything after a `--` is passed to gauge):
46+Anything after a `--` is passed to gauge, it basically selects all benchmarks
47+starting with `Prelude.Serial/o-1-space` prefix.
48+
49+Run a specific benchmark in `Prelude.Serial` suite:
3250
3351```
34−$ ./bench.sh --benchmarks base -- StreamD
52+$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space/generation/unfoldr
3553```
3654
37−## Comparing benchmarks
55+## Comparing results for regression
3856
3957To compare two sets of results, first run the benchmarks at the baseline
4058commit:
@@ -75,26 +93,30 @@ Note that the above may not always work because the script and the benchmarks
7593themselves might have changed across the commits. The `--append` method is more
7694reliable to compare.
7795
78−## Available Benchmarks
96+## Comparing benchmark suites
97+
98+First see the available benchmark suites:
7999
80−The benchmark names that you can use when running `bench.sh`:
100+```
101+$ ./bench.sh --benchmarks help
102+```
81103
82−* `base`: a benchmark that measures the raw operations of the basic streams
83− `StreamD` and `StreamK`.
104+You will see some benchmark suites end with `_cmp`, these are comparison
105+groups. If you run a comparsion group benchmark, comparison of all the
106+benchmark suites in that group will be shown in the end. For example to compare
107+all array benchmark suites:
84108
85−* `linear`: measures the non-monadic operations of serial streams
86−* `linear-async`: measures the non-monadic operations of concurrent streams
87−* `linear-rate`: measures the rate limiting operations
88−* `nested`: measures the monadic operations of all streams
89−* `all`: runs all of the above benchmarks
109+```
110+$ ./bench.sh --benchmarks array_cmp
111+```
90112
91113## Reporting without measuring
92114
93115You can use the `--no-measure` option to report the already measured results in
94116the benchmarks results file. A results file may collect an arbitrary number of
95117results by running with `--append` multiple times. Each benchmark has its own
96−results file, for example the `linear` benchmark has the results file at
97−`charts/linear/results.csv`.
118+results file, for example the `Prelude.Serial` benchmark has the results file at
119+`charts/Prelude.Serial/results.csv`.
98120
99121You can also manually edit the file to remove a set of results if you like or
100122to append results from previously saved results or from some other results