Filter for test/Benchmarks (#8391)

With herein proposed change one can pass an optional filter to `enso --run test/Benchmarks` to execute only groups and specs that contain given string in its name.
This commit is contained in:
Jaroslav Tulach 2023-11-27 16:27:12 +01:00 committed by GitHub
parent b51dfe5a5b
commit c6eb61a055
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 38 additions and 10 deletions

View File

@ -60,14 +60,27 @@ environment variable before running any benchmarks.
### Running standalone
A single source file in the project may contain multiple benchmark definitions.
If the source file defines `main` method, we can evaluate it the same way as any
other Enso source file, for example via
`runEngineDistribution --in-project test/Benchmarks --run <source file>`. The
harness within the project is not meant for any sophisticated benchmarking, but
rather for quick local evaluation. See the `Bench.measure` method documentation
for more details. For more sophisticated approach, run the benchmarks via the
JMH launcher.
There is a universal launcher that enlists and executes all available benchmarks
in `test/Benchmarks` project. Run it with
```bash
enso$ runEngineDistribution --run test/Benchmarks
```
command. The launcher accepts additional `filter` argument which allows one to
select a benchmark of one's choice by checking for substrings in group or
benchmark name. For example:
```bash
enso$ runEngineDistribution --run test/Benchmarks New_Vector
```
runs all the benchmarks that have `New_Vector` in their name.
The harness within the project is not meant for any sophisticated benchmarking,
but rather for quick local evaluation. See the `Bench.measure` method
documentation for more details. For more sophisticated approach, run the
benchmarks via the JMH launcher.
### Running via JMH launcher

View File

@ -89,8 +89,23 @@ all_benchmarks =
builder.to_vector
main =
benchmarks = all_benchmarks
main filter=Nothing =
benchmarks = if filter.is_nothing then all_benchmarks else
no_nothing x = x.is_nothing.not
filter_benchmarks (b:Bench) = case b of
Bench.All groups ->
vec = groups.map filter_benchmarks . filter no_nothing
if vec.is_empty then Nothing else
Bench.All vec
Bench.Group n opts specs -> if n.contains filter then b else
vec = specs.map filter_benchmarks . filter no_nothing
if vec.is_empty then Nothing else
Bench.Group n opts vec
Bench.Spec n _ -> if n.contains filter then b else Nothing
all_benchmarks.map filter_benchmarks . filter no_nothing
total_specs = benchmarks.map .total_specs . fold 0 (+)
IO.println "Found "+benchmarks.length.to_text+" benchmark suites, containing "+total_specs.to_text+" specs in total."
estimated_duration = benchmarks.map .estimated_runtime . fold Duration.zero (+)