58cb9ca · streamly · git.iko.soy

git.iko.soy/streamly

on master help sign in

Move the bench.sh script to "bin" dir

58cb9caHarendra Kumar6 years ago

benchmark/README.md+17−19

compare with master

unifiedsplit

@@ -66,19 +66,17 @@ $ cabal bench streamly-benchmarks

67## Building and Running Benchmarks with bench.sh

69−`bench.sh` script at the root of the repo is the top level driver for running

69+`<streamly repo>/bin/bench.sh` script is the top level driver for

70−benchmarks. It runs the requested benchmarks and then creates a report from the

70+running benchmarks. It runs the requested benchmarks and then creates a

71−results using the `bench-show` package. Try `bench.sh --help` for available

71+report from the results using the `bench-show` package. Try `bench.sh

72−options to run it.

72+--help` for available options to run it.

74## bench.sh: Quick start

76−You must be in the repo root directory to run these commands.

77−

78Run the default benchmark suites:

76Run the default benchmark suites:

80```

78```

81−$ ./bench.sh --quick

79+$ bench.sh --quick

82```

80```

84You can remove the `--quick` option to run benchmark with lower speed but

82You can remove the `--quick` option to run benchmark with lower speed but

@@ -88,25 +86,25 @@ the accuracy a bit.

88Show available benchmark suites:

86Show available benchmark suites:

90```

88```

91−$ ./bench.sh --benchmarks help

89+$ bench.sh --benchmarks help

92```

90```

94Run all benchmark suites in the `serial_grp` group:

92Run all benchmark suites in the `serial_grp` group:

96```

94```

97−$ ./bench.sh --benchmarks serial_grp

95+$ bench.sh --benchmarks serial_grp

98```

96```

100Run `Prelude.Serial` and `Data.Parser` benchmark suites:

98Run `Prelude.Serial` and `Data.Parser` benchmark suites:

101

102```

100```

103−$ ./bench.sh --benchmarks "Prelude.Serial Data.Parser"

101+$ bench.sh --benchmarks "Prelude.Serial Data.Parser"

104```

102```

105

103

106Run all O(1) space complexity benchmarks in `Prelude.Serial` suite:

104Run all O(1) space complexity benchmarks in `Prelude.Serial` suite:

107

105

108```

106```

109−$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space

107+$ bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space

110```

108```

111

109

112Anything after a `--` is passed to the benchmark executable,

110Anything after a `--` is passed to the benchmark executable,

@@ -116,7 +114,7 @@ it basically selects all benchmarks starting with

116Run a specific benchmark in `Prelude.Serial` suite:

114Run a specific benchmark in `Prelude.Serial` suite:

117

115

118```

116```

119−$ ./bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space/generation/unfoldr

117+$ bench.sh --benchmarks Prelude.Serial -- Prelude.Serial/o-1-space/generation/unfoldr

120```

118```

121

119

122Run a benchmark directly instead of running it through `bench.sh`:

120Run a benchmark directly instead of running it through `bench.sh`:

@@ -136,14 +134,14 @@ To compare two sets of results, first run the benchmarks at the baseline

136commit:

134commit:

137

135

138```

136```

139−$ ./bench.sh

137+$ bench.sh

140```

138```

141

139

142And then run with the `--append` option at the commit that you want to compare

140And then run with the `--append` option at the commit that you want to compare

143with the baseline. It will show the comparison with the baseline:

141with the baseline. It will show the comparison with the baseline:

144

142

145```

143```

146−$ ./bench.sh --append

144+$ bench.sh --append

147```

145```

148

146

149Append just adds the next set of results in the same results file. You can keep

147Append just adds the next set of results in the same results file. You can keep

@@ -157,19 +155,19 @@ while.

157You can use `--compare` to compare the previous commit with the head commit:

155You can use `--compare` to compare the previous commit with the head commit:

158

156

159```

157```

160−$ ./bench.sh --compare

158+$ bench.sh --compare

161```

159```

162

160

163To compare the head commit with some other base commit:

161To compare the head commit with some other base commit:

164

162

165```

163```

166−$ ./bench.sh --compare --base d918833

164+$ bench.sh --compare --base d918833

167```

165```

168

166

169To compare two arbitrary commits:

167To compare two arbitrary commits:

170

168

171```

169```

172−$ ./bench.sh --compare --base d918833 --candidate 38aa5f2

170+$ bench.sh --compare --base d918833 --candidate 38aa5f2

173```

171```

174

172

175Note that the above may not always work because the script and the benchmarks

173Note that the above may not always work because the script and the benchmarks

@@ -181,7 +179,7 @@ reliable to compare.

181First see the available benchmark suites:

179First see the available benchmark suites:

182

180

183```

181```

184−$ ./bench.sh --benchmarks help

182+$ bench.sh --benchmarks help

185```

183```

186

184

187You will see some benchmark suites end with `_cmp`, these are comparison

185You will see some benchmark suites end with `_cmp`, these are comparison

@@ -190,7 +188,7 @@ benchmark suites in that group will be shown in the end. For example to compare

190all array benchmark suites:

188all array benchmark suites:

191

189

192```

190```

193−$ ./bench.sh --benchmarks array_cmp

191+$ bench.sh --benchmarks array_cmp

194```

192```

195

193

196## Reporting without measuring

194## Reporting without measuring

bench.sh → bin/bench.shrenamed+4−2

@@ -97,7 +97,7 @@ ALL_BENCH_GROUPS="\

97# Script

98#------------------------------------------------------------------------------

100−BENCH_SH_DIR=$(dirname $0)

100+BENCH_SH_DIR=$(cd `dirname $0`; pwd)

101

102list_benches () {

103 echo "Individual benchmarks:"

@@ -330,7 +330,7 @@ run_bench () {

330 run_verbose $bench_prog $SPEED_OPTIONS \

331 --csvraw=$output_file \

332 -v 2 \

333− --measure-with "$BENCH_SH_DIR/bin/bench-exec-one.sh" \

333+ --measure-with "$BENCH_SH_DIR/bench-exec-one.sh" \

334 $GAUGE_ARGS || die "Benchmarking failed"

335}

336

@@ -413,6 +413,8 @@ run_reports() {

413# Execution starts here

414#-----------------------------------------------------------------------------

415

416+cd $BENCH_SH_DIR/..

417+

416DEFAULT_BENCHMARKS="$all_grp"

418DEFAULT_BENCHMARKS="$all_grp"

417DEFAULT_FIELDS="allocated bytescopied cputime"

419DEFAULT_FIELDS="allocated bytescopied cputime"

418ALL_FIELDS="$FIELDS time cycles utime stime minflt majflt nvcsw nivcsw"

420ALL_FIELDS="$FIELDS time cycles utime stime minflt majflt nvcsw nivcsw"

streamly.cabal+1−1

@@ -106,7 +106,6 @@ extra-source-files:

106 MAINTAINING.md

107 README.md

108 appveyor.yml

109− bench.sh

110 benchmark/*.hs

109 benchmark/*.hs

111 benchmark/README.md

110 benchmark/README.md

112 benchmark/Streamly/Benchmark/Data/*.hs

111 benchmark/Streamly/Benchmark/Data/*.hs

@@ -120,6 +119,7 @@ extra-source-files:

120 benchmark/lib/Streamly/Benchmark/*.hs

119 benchmark/lib/Streamly/Benchmark/*.hs

121 benchmark/lib/Streamly/Benchmark/Common/*.hs

120 benchmark/lib/Streamly/Benchmark/Common/*.hs

122 benchmark/streamly-benchmarks.cabal

121 benchmark/streamly-benchmarks.cabal

122+ bin/bench.sh

123 bin/bench-exec-one.sh

124 bin/build-lib.sh

125 bin/mk-hscope.sh