mirror of
https://github.com/digital-asset/daml.git
synced 2024-09-20 09:17:43 +03:00
6e48abc793
This should be merged after #6080. This PR adds a patch (and consequently updates the `ci/cron/perf/compare.sh` script) to apply the same logical change as #6080 on top of the baseline commit, so our performance comparison remains "apples to apples". I am well aware that managing patches is not going to be a great way forward. The rate of changes on the benchmark seems to be slow enough that this is good enough for now, but should we change the benchmark more often and/or want to add new benchmarks, a better approach would be to handle the changes at the Scala level. That is: - Create a "rest of the world" (world = Speedy, its compiler, and all of the associated types) interface that benchmarks would depend on, rather than depend directly on the rest of the codebase. - Create two implementations of that interface, one that compiles against the current state of the world, and one that compiles against the baseline. - Change the script to load the relevant implementation, and then run all the benchmarks as-is, with no match necessary. CHANGELOG_BEGIN CHANGELOG_END
35 lines
1.2 KiB
Bash
Executable File
35 lines
1.2 KiB
Bash
Executable File
# Copyright (c) 2020 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
#!/usr/bin/env bash
|
|
|
|
set -euo pipefail
|
|
|
|
BASELINE=$1
|
|
|
|
measure() {
|
|
local out=$(mktemp -d)/out.json
|
|
bazel run daml-lf/scenario-interpreter:scenario-perf -- -rf json -rff $out >&2
|
|
cat $out | jq '.[0].primaryMetric.score'
|
|
}
|
|
|
|
main() {
|
|
local current=$(git rev-parse HEAD)
|
|
|
|
git checkout $BASELINE >&2
|
|
git show ${current}:ci/cron/perf/CollectAuthority.scala.patch | git apply
|
|
local baseline_perf=$(measure)
|
|
|
|
git checkout -- daml-lf/scenario-interpreter/src/perf/benches/scala/com/digitalasset/daml/lf/speedy/perf/CollectAuthority.scala
|
|
git checkout $current >&2
|
|
local current_perf=$(measure)
|
|
|
|
local speedup=$(printf "%.2f" $(echo "$baseline_perf / $current_perf" | bc -l))
|
|
local progress_5x=$(printf "%05.2f%%" $(echo "100 * l($speedup) / l(5)" | bc -l))
|
|
local progress_10x=$(printf "%05.2f%%" $(echo "100 * l($speedup) / l(10)" | bc -l))
|
|
|
|
echo '{"current-perf": '$current_perf', "baseline-perf": '$baseline_perf', "speedup": "'$speedup'x", "progress_towards_5x": "'$progress_5x'", "progress_towards_10x": "'$progress_10x'", "current-sha": "'$current'", "baseline-sha": "'$BASELINE'"}'
|
|
}
|
|
|
|
main
|