* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
* Fix contains check in ConcurrentCompiledPackages
`contains` searches for a _value_ not a _key_. We have a package id
here in both cases which is clearly a key.
I am not exactly clear on what exactly happens if you hit this bug. I
believe it’s just a performance issue so probably hard to write a test
for.
I did take a brief look at whether we can make this
typesafe (`contains` accept an `Object` which is how this typechecks)
and apparently calling `asScala` and then using
`scala.collection.concurrent.Map` should work but I don’t know if this
has performance implications or if there is another reason why we
didn’t do this. Happy to make the change if someone tells me this is
the right thing to do.
changelog_begin
changelog_end
* Switch to Scala’s concurrent map which doesn’t pretend types are bad
changelog_begin
changelog_end
At the moment, JMH seems happy to just swallow exceptions and consider
the benchmark done, which makes it produce inaccurate speed results and
lets errors slip through to master. This makes unexpected errors in the
benchmark a hard stop.
This is not a complete solution: ideally there would be a way to just
tell jmh to abort on uncaught exceptions. However, I don't seem to be
able to find any relevant documentation on how to do that.
CHANGELOG_BEGIN
CHANGELOG_END
During some refactoring we forgot to save the initial expression to
evaluate for the machine during benchmarking. This PR fixes the issue.
It also make the error messages a bit more descriptive so that we can
actually debug this.
A test to make sure issues like this one don't get through CI again is
worked on by @gary-verhaegen-da in a separate PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Consistently display stakeholders for key visibility errors
fixes#6404
As pointed out by Bernhard in #6404, the previous behavior was pretty
weird. If the committer was only a divulgee, we only displayed
stakeholders. If the committer was neither a stakeholder nor a
divulgee, we displayed stakeholders + parties the contract has been
divulged to. Given that only stakeholders can do lookups it makes much
more sense to display them consistently which is what this PR
achieves. I’ve also renamed “disclosed to” to “stakeholders” to make
it very explicit what is shown there.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* fmt
changelog_begin
changelog_end
* lalala
changelog_begin
changelog_end
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
fixes#6403
I am not entirely sure why I thought that using `missingWith` makes
sense here but it clearly doesn’t make sense and resulted in a pretty
bad bug where a transaction both succeeded via `submit` as well as
failed via `submitMustFail` which is clearly the wrong thing to do.
This PR fixes this issue and introduces a `notVisibleWith` function
that does the right thing. I’ve also added some comments and an extra
assertion to clarify things a bit.
changelog_begin
changelog_end
In this PR we cleanup the constructor for the speedy Machine.
* We remove the `case` keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
+ one generic builder `Machine.apply`,
+ scenario specific builder,
CHANGELOG_BEGIN
CHANGELOG_END
* use Profile.Label newtype for typechecked union instead of AnyRef
- includes port of some of interpreter
- demonstrating its efficacy is the compiler error in this commit:
daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala:267: error: type mismatch;
found : com.daml.lf.speedy.SExpr.SEBuiltinRecursiveDefinition
required: com.daml.lf.speedy.Profile.Label
(which expands to) com.daml.lf.speedy.Profile.LabelModule.Module.T
withLabel(ref, ref)
^
What was likely intended was to write `ref.ref` here; that is the assumption
the `Event#label` rendering function makes, anyway. Now, we type-check that
the labelling matches the renderer.
* let Profile make arbitrary Labels
* fix null and missing `.ref` calls for labelling
* one more null => LabelUnset
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* implicitNotFound message never used
* ritual offering of the dot and parens
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* LF: rename library transaction-scalacheck to transaction-test-lib
CHANGELOG_BEGIN
CHANGELOG_END
* move files in com/daml
* missing change in release/artifacts.yaml
* remove 'com/dam' from the path
We split the object com.daml.lf.types.ledger in three:
- one part in `com.daml.lf.ledger.` (in //daml-lf/transaction) for the part relative to EventId (shares between scenario service and sandbox)
- one part in `com.daml.lf.ledger.` (in //daml-lf/interpreter) for the part common to Blinding and Scenario
- one part in `com.daml.lf.scenario.` (in //daml-lf/interpreter) for the part specific to Scenario
fixes#6260
CHANGELOG_BEGIN
CHANGELOG_END
This PR adds some DAML speed benchmarks which focus on the computational aspects of DAML.
The first benchmark is `nfib`. The speed is reported in nfibs/micro-second.
The second benchmark is `json-parser`. We have a short pipeline: JSON AST is constructed
to represent an arithmetic expression. The AST is converted to its string representation.
The JSON string is then parsed back to AST using the JSON parser (which is defined using
parser combinators defined in the benchmark code). Finally the arithmetic expression
embedded in the JSON AST is evaluated. We report the speed in k-chars/second.
The speed tests are designed to be quick and easy to run. Both tests scale exponentially
in their integer argument, and so are easy to tune so each iteration takes about half a
second. The are run like this:
```
bazel run daml-lf/interpreter/perf:nfib
bazel run daml-lf/interpreter/perf:speed-json-parser
```
For interest, the speeds I see on my dev machine are:
- nfib: 1.35 nfibs/us
- json-parser: 27 k/s
changelog_begin
changelog_end
This PR fixes a bug where the profiler wrote open events for functions
_before_ evaluating their arguments. However, in a call-by-value
language such as DAML, arguments are evaluated before entering a
function and the profiler should reflect that.
This PR also adds a regression test.
CHANGELOG_BEGIN
CHANGELOG_END
In order to make the flamegraphs easier to consume by humans, we change
the profiler output in the following ways:
1. Don't print package ids anymore. They are not particularly useful
but cause a lot of noise.
2. Remove a few useless angle bracket and move the printed names of
DAML-LF closer to their surface level names.
3. Unmangle identifiers on a best effort basis.
4. Give the profiles shorter names such that they don't occupy the
whole screen and leave some space for the navigation buttons of the
Speedscope UI.
CHANGELOG_BEGIN
CHANGELOG_END
* Optimize the execution of Saturated Builtin Applications in Speedy.
We special case applications where the expression in function-position is a builtin operator, and the number of arguments matches the arity of the builtin. The special-case detection is done at compile time, and allows for more efficient runtime execution, specifically:
- We don't need to construct an `SPAP` value, only to immediately deconstruct/enter it.
- We don't need to do arity checking at runtime, with special case handling for _partial-_ and _over-_ applications.
The change gives about 3% speedup.
changelog_begin
changelog_end
* improve doc comments & make class names more descriptive
* share code for evaluating arguments
* improve name: SEAppSaturatedBuiltinFun
* optimize over-applied builtin function applications
* fix bug in the refactoring which introduced evaluateArguments
* replace NodeExercises#controllers with controllersDifferFromActors
* remove controllers from ActorMismatch and scenario service exercise
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove reserved ID #s in scenario-service grpc
As discussed, we don't need to worry about
version mismatches.
Co-authored-by: Remy <remy.haemmerle@daml.com>
* DAML profiler: Use non-blocking IO for writing profiles
As suggested by @SamirTalwar-DA.
CHANGELOG_BEGIN
CHANGELOG_END
* Follow Samir's advice even closer
CHANGELOG_BEGIN
CHANGELOG_END
* Special case atomic expressions in the functional-position of applications.
We regard builtins, values and variables as atomic. The special case detection is done at compile time; at run time, the atomic case avoids one push/pop on the continuation stack.
For the bench example, about 2/3 of the applications performed at run-time, fall into the special case, leading to an overall reduction in about 15% of the steps taken by the Speedy CEK machine.
This change gives a 3% to 4% performance improvement.
changelog_begin
changelog_end
* address review comments
Previously, we just crashed the scenario service instead of throwing a
proper scenario error. This meant that you had to look at the
debugging output to figure out what is going wrong. This is both a
shitty UX and also inconsistent with how we handle this for fetch and
exercise on contract ids that are not visible. This PR adds a new
error type that matches the one for invisible contract ids.
changelog_begin
- [DAML Studio] Fetches and exercises of contract keys associated with
contracts not visible to the submitter are now handled properly
instead of showing a low-level error.
changelog_end
fixes#5903
* Expose the DAML profiler in Sandbox Classic
This PR adds an option `--profile-dir` to Sandbox Classic which takes
a directory as its argument. When this option is used, all command
submissions to the Sandbox are run in profiling more and the resulting
profiling information is written to the directory specified via
`--profile-dir`. See `/ledger/sandbox/README.md#Profiling` for further
details on this.
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt DAML-LF REPL
CHANGELOG_BEGIN
CHANGELOG_END
* remove boilerplate from GenNode.map3
- There is no need to enumerate the unchanged fields, because if you
miss one, you get a type error. Unfortunately foreach3 cannot benefit
from the same typechecking.
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
* add law checking for SValue Ordering
CHANGELOG_BEGIN
CHANGELOG_END
* nested example for randomComparableValues
* match comparableAbsCoidsGen -> comparableCoidsGen renaming
* register scala-collection-compat with java deps list
* add scala-collection-compat to http-json deps
* remove breakOut throughout http-json, replaced with view/to or iterator/to
* use scala 2.13-style `to` calls in http-json
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* use 2.13-style to in lf-value-json
* some fused size comparisons
* remove low-hanging breakOuts in daml-lf
* regenerate maven_install.json for scala-collection-compat
* regenerate maven_install.json for scala-collection-compat
* regenerate maven_install.json for scala-collection-compat
* equalz Scalatest matcher in new daml-lf/scalatest-tools library
* equalz typing tests
* a 'should' replacing design
* a 'MatcherFactory1' design
- this fails because the TC parameter should be a type member to avoid
scala/bug#5075 but it is not
* MatcherFactory1 with chained Lub+Equal typeclass
- requires partial-unification at point of use, which is not great
* LubEqual's extra tparam is probably unneeded
* better LtEqual
* demonstrate that HK LubEqual's resolve with DMT should + MatcherFactory
* remove unneeded 3rd param from LubEqual, again
* update dependency specs and license headers
* allow use with should, shouldNot in some cases, preserving the shouldx/shouldNotx alternatives
* move Equalz to libs-scala/scalatest-utils
* rename bzl targets and place in com.daml.scalatest package
* add scalatest-utils to release
* move *SpecCheckLaws, Unnatural to scalatest-utils
* missed scalacheck dep in scalatest-utils
* downstreams of *SpecCheckLaws now get them from scalatest-utils
* test equal-types case as well
* update LF documentation
CHANGELOG_BEGIN
CHANGELOG_END
* whitespace error
* Disentangle the execution of closures and builtin functions.
Both closures and builtins take arguments, but only closures have free variables.
Consequently, we split the machine component `frame` into `frame` and `actuals`.
Allowing them to be managed independently, and simplifying lookup for `SELocF` and `SELocA`.
The `KFun` continuation is split into 3 variants: `KFun`, `KBuiltin` and `KPap`.
These handle the execution of three different cases of function application:
- execution of a user function (a closure)
- execution of a builtin
- execution of a partial application (builtin or closure, it doesn't matter)
The choice of which continuation to push is made in the `KArg` continuation when the
function-expression has been evaluated and we have discovered what it is - builtin or
closure - and we know it's arity.
The prior code made some related decisions (for example, is this an over-application) but
left other choices to `KFun` (code was: `enterFullyAppliedFunction`). This required that
the `KFun` continuation had to stash the entire `Prim` (closure/builtin) and `arity`, only
to then re-examine it when the continuation was entered.
Now, all decisions are made within `KArg`, making the structure and execution of the three
new continuations much simpler. In particular, the `machine.frame` is only set when we
enter `KFun`.
All continuations which need to preserve their environment, now save both frame & actuals,
along with the env-stack-size. And `restoreEnv` is adapted accordingly.
changelog_begin
changelog_end
* improve variable name
* with SomeArrayEquals
The aim of the `CollectAuthority` benchmark is to track performance improvements made to the
DAML compiler & the speedy interpreter. Unfortunately the benchmark was spending at least
20% of the time being benchmarked outside of the speedy machine execution code, and
instead interacting with the ledger. We would like to minimize this as much as possible.
The solution is to cache the responses from the ledger made during the setup() run, and
replay them during the benchmark `run()`s. To perform the caching, we no longer make use
of the scenario-interpreter, but instead have our own simplified runner, specialized for
this benchmark, and managing the cache.
using the cache speeds up the benchmark by a factor of about x1.25 (Which is the origin of
the claim above about 20% outside of speedy execution).
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
95.188 ±(99.9%) 0.410 ms/op [Average]
(min, avg, max) = (94.177, 95.188, 95.890), stdev = 0.472
CI (99.9%): [94.779, 95.598] (assumes normal distribution)
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
75.881 ±(99.9%) 0.288 ms/op [Average]
(min, avg, max) = (75.077, 75.881, 76.491), stdev = 0.332
CI (99.9%): [75.593, 76.169] (assumes normal distribution)
Because of how we run perf benchmark on CI -- by running the benchmark for now vs. some
fixed time in the past -- it ought to be fine to change the benchmark like this, although
we might need some help from Gary!
As well as improving the focus of this benchmark going forwards, we should also gain a
retrospective improvement on the speedup work already committed, since they will no longer
we dragged down by time which is outside of our control.
changelog_begin
changelog_end
