This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Port the rest //daml-lf/... to Scala 2.13
Draw the rest of the owl
changelog_begin
changelog_end
* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
We use the constants defined in LanguageVersion/TransactionVersions instead
repeating the somewhat magic constants "6", "7", "8", "10" and "dev" in multiple places?
CHANGELOG_BEGIN
CHANGELOG_END
We drop the distinction (at the type level) of Dev and Stable language
version. The two main reason that motivate this choice:
* we never really used this distinction.
* we want to add the concept of "preview" version (which is neither Dev nor Stable)
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.
* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.
* Bazel: Factor out Java/Scala protobuf class generation into a helper.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/archive: Use `proto_jars`.
* Bazel: Remove the visibility modifier from `proto_jars`.
It's too confusing. Just make everything public.
* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.
* Bazel: Add comments to the various parts of `proto_jars`.
* daml-assistant: Do unpleasant things with `location` in Bazel.
* Cache computation of top-level values at definition level
Earlier the computation of a top-level value was only cached at the
use-site in SEVal. This introduces SDefinition which contains the same
mechanism as SEVal to cache the computation.
As expected this does not impact performance much:
before: CollectAuthority.bench //daml-lf/scenario-interpreter/CollectAuthority.dar CollectAuthority:test avgt 40 44.267 ± 0.728 ms/op
after: CollectAuthority.bench //daml-lf/scenario-interpreter/CollectAuthority.dar CollectAuthority:test avgt 40 43.693 ± 0.702 ms/op
What this does have a significant impact is on reducing the number of distinct
SValues for things like type class dictionaries etc, so that now we have one
SValue per dictionary rather than one per SEVal.
CHANGELOG_BEGIN
CHANGELOG_END
* Address code review
* Fix speedy tests
* Speedy: group compiler parameters into a Config case class.
Speedy compiler is parametrized by several option flags. Those flags
are passed through different calls (in particular in the instance of
`CompiledPackages`.
This PR group all this configuration parameters into a case class to
pass the option in a cleaner way.
CHANGELOG_BEGIN
CHANGELOG_END
A `Struct[+X]` is a list of pair `(FieldName, X)` ordered by first
component.
We use this data-structure to represent TStruct LF type.
It will be used in upcomming PRs to sorted fields in Struct values.
CHANGELOG_BEGIN
CHANGELOG_END
* add -Ywarn-unused to all scalac options
* remove some unused arguments
* remove some unused definitions
* remove some unused variable names
* suppress some unused variable names
* changeExtension doesn't use baseName
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* work around no plugins in scenario interpreter perf tests
* remove many more unused things
* remove more unused things, restore some used things
* remove more unused things, restore a couple signature mistakes
* missed import
* unused argument
* remove more unused loggingContexts
* some unused code in triggers
* some unused code in sandbox and kvutils
* some unused code in repl-service and daml-script
* some unused code in bindings-rxjava tests
* some unused code in triggers runner
* more comments on silent usages
- suggested by @cocreature; thanks
* fix missing reference in TestCommands
* more unused in triggers
* more unused in sandbox
* more unused in daml-script
* more unused in ledger-client tests
* more unused in triggers
* more unused in kvutils
* more unused in daml-script
* more unused in sandbox
* remove unused in ledger-api-test-tool
* suppress final special case for codegen unused warnings
.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
* one more unused in daml-script
* special scaladoc rules may need silencer, too
* unused in compatibility/sandbox-migration
* more commas, a different way to `find`
- suggested by @remyhaemmerle-da; thanks
* LF: Simplify inference of output transaction/value version.
As part of #5164, we simplify the way transaction and value versions
are inferred for the engine output.
The versioning of a transaction tr is done as follow:
* Let tvMin be the minimal transaction version allowed by the DAML
engine configuration
* Let tvMax be the maximal transaction version allowed by the DAML
engine configuration
* Let n₁, ..., nₘ be the nodes of tr.
* Let pkgᵢ be the package of the template associated to the node nᵢ
* Let lvᵢ be the maximal language version that pkgᵢ uses (directly or
through its dependencies)
* let tvᵢ be the maximal transaction version supported by all engines
that support both lvᵢ and tvMin
* Let vvᵢ be the maximal value version supported by all engines that
support tvᵢ.
* Let tv be the maximal transaction version between tv₁, ..., tvₘ.
* If tv is not greater than tvMax
+ then
- Version the values of each node nᵢ according vvᵢ
- Version tr according tv
+ fail otherwise
CHANGELOG_BEGIN
CHANGELOG_END
* Revert ANF changes and add a testcase for evaluation order
After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.
Therefore, this PR reverts the following commits:
1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee
Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably
1. https://github.com/digital-asset/daml/pull/6576 which was easy to
resolve and the change to return SEValue instead of SExpr is still
nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
some changes since the constructors changed. If you want to review
those changes in detail (they are pretty straightforward so not too
important), it’s probably easiest to check out this PR and run
```
git diff 2cd2a8f2a8
daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
```
to see the diff to the parent commit of the first commit that
introduced ANF.
changelog_begin
changelog_end
* ANF transformation in Speedy.
The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.
changelog_begin
changelog_end
* improve comment
* inline functions relocateA/L
* remove comment about scalafmt
* remove commented out alterative def for transformLet1
* improve code by adding incr methods to DepthA/E
* remove (n == 0) special case in trackBindings
* clarify comment further
* improve validate/go to not consume stack for deeply right-nested let-expressions
* address comments from Remy: be private; use final case case; etc
* rename to unsafeCompilationPipeline
* add back some trailing commas
* remove commented-out debug line
* improve comment
* remove dev/debug code in compilationPipeline
* remove commented out code in SEAppGeneral.execute
* undo unrelated code improvement in SValue.scala
* fix compile. object Anf cannot be private
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
In this PR we cleanup the constructor for the speedy Machine.
* We remove the `case` keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
+ one generic builder `Machine.apply`,
+ scenario specific builder,
CHANGELOG_BEGIN
CHANGELOG_END
We split the object com.daml.lf.types.ledger in three:
- one part in `com.daml.lf.ledger.` (in //daml-lf/transaction) for the part relative to EventId (shares between scenario service and sandbox)
- one part in `com.daml.lf.ledger.` (in //daml-lf/interpreter) for the part common to Blinding and Scenario
- one part in `com.daml.lf.scenario.` (in //daml-lf/interpreter) for the part specific to Scenario
fixes#6260
CHANGELOG_BEGIN
CHANGELOG_END
* Expose the DAML profiler in Sandbox Classic
This PR adds an option `--profile-dir` to Sandbox Classic which takes
a directory as its argument. When this option is used, all command
submissions to the Sandbox are run in profiling more and the resulting
profiling information is written to the directory specified via
`--profile-dir`. See `/ledger/sandbox/README.md#Profiling` for further
details on this.
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt DAML-LF REPL
CHANGELOG_BEGIN
CHANGELOG_END
* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
Choices for `stacktracing` are `NoStackTrace` / `FullStackTrace`.
Adapt code so the selection is made by the original caller:
- `engine`
- `scenario-service`
- `repl-service`
- `daml-script` runner
etc
Currently, all callers pass `FullStackTrace` (the existing behaviour), except for the
exploration dev-code: `daml-lf/interpreter/perf/src/main/scala/com/daml/lf/explore`.
The idea is that once this control is in place, we can discuss if we can change how we
might expose it to the user, and/or perhaps change the default behaviour to have
`stacktracing` disabled.
changelog_begin
changelog_end
* Implement a simple profiler for DAML scenarios
The profiler runs a single scenario and records timing information when
each function (and some other closures) are entered and left. The
resulting information can be visualized as a flamegraph using
[speedscope](https://www.speedscope.app/).
The profiler works by instrumenting the CEK machine at the heart of
DAML Engine. Unfortunetaly, this causes a very small overhead on
non-profiling runs too. However, in my benchmarks I could not measure
any significant impact on the overall runtime at all. More precisely,
the overhead is as follows:
Every closure now has an additional field called `label`. In
non-profiling runs this field is always set to `null`. This field needs
to be allocated, copied whenever we copy a closure and scanned during
garbage collection. Additionally, whenever we enter a closure, we check
this field and whenever it is _not_ `null`, i.e. never during
non-profiling runs, we record an "open event" and set up a hook for the
corresponding "close event". Thus, the additional cost during
non-profiling runs are a single pointer comparison and a jump beyond
the "then branch".
Since this is still very much in active development, there are no
documentation, other than an entry in a README, and no tests yet. They
will come before we promote this. However, the UX will look very
different then since we already have plans to significantly change it.
CHANGELOG_BEGIN
CHANGELOG_END
* Run scalafmt
* Make profiling argument to PureCompiledPackges optional
* Fix a bunch of tests
CHANGELOG_BEGIN
CHANGELOG_END
* scalafmt is so annoying
CHANGELOG_BEGIN
CHANGELOG_END
* Apply simple suggestions
CHANGELOG_BEGIN
CHANGELOG_END
* Simplify and clarify the public interface to Speedy.
- Remove `isFinal`. A client just uses `run()`.
- Remove `toSValue`. The value in available in `SResultFinalValue(v: SValue)`.
- A client never directly access the `.ctrl` (or `.returnValue`) components.
- A client may use `setExpressionToEvaluate(expr)` to evaluate a new expression on an existing machine.
changelog_begin
changelog_end
* remove while loop which executes just once
* avoid unnecessary mutation when running speedy
Remove the `Ctrl` trait and separate `Machine.ctrl: Ctrl` into `Machine.ctrl: SExpr` and `Machine.returnValue: SValue` instead. This allows for avoiding dynamic dispatch on `ctrl` and instead allows for checking a pointer for `null` to decide if we have an expression that needs further break-down or a return value ready to be passed to the next continuation.
To make this check really only a pointer comparison we also needed to remove the abomination of "fully applied partially applied primitives". In order to achieve this, we check whether a PAP will be fully applied afterward when applying the last argument.
On the `collect-authority` benchmark, this increases throughput by around 13%, on another more computation heave benchmark by about 21%.
`collect-authority` benchmark on `master`:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
112.361 ±(99.9%) 1.965 ms/op [Average]
(min, avg, max) = (107.047, 112.361, 120.745), stdev = 3.493
CI (99.9%): [110.396, 114.326] (assumes normal distribution)
```
`collect-authority` benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
98.196 ±(99.9%) 1.933 ms/op [Average]
(min, avg, max) = (91.580, 98.196, 105.478), stdev = 3.436
CI (99.9%): [96.263, 100.129] (assumes normal distribution)
```
computation heavy benchmark on master
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
44.030 ±(99.9%) 0.742 ms/op [Average]
(min, avg, max) = (42.124, 44.030, 46.781), stdev = 1.319
CI (99.9%): [43.289, 44.772] (assumes normal distribution)
```
computation heavy benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
36.222 ±(99.9%) 0.580 ms/op [Average]
(min, avg, max) = (34.897, 36.222, 39.787), stdev = 1.031
CI (99.9%): [35.643, 36.802] (assumes normal distribution)
```
changelog_begin
changelog_end
Speedy: run() dont step()
- Running the Speedy machine with `run()` instead of `step()`
- Remove: `SResultContinue`
- Add: `SResultFinalValue(_)`
We change the top level control of Speedy: from machine.step() to machine.run, with the control of stepping while the machine returns SResultContinue moved into speedy itself. (And so SResultContinue is removed in favour of SResultFinalValue.) The main advantage of this approach is that the tight while loop can be moved inside the exception handler, rather than having to wrap the handler every step.
changelog_begin
changelog_end
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* building a GenMap generator
* move Party Order to accessible place; test whether Name Equal is still in use
* add Order instance for SortedLookupList
* switch to Map for genMap's Inj; more Order instances
* remove all Order from TypedValueGenerators
* Revert "add Order instance for SortedLookupList"
This reverts commit 03a59a8249.
* moving the Equal instance means scenario-interpreter no longer direct-deps scalaz
* add a test using TypedValueGenerators GenMap
* remove Party Order
* refmt bazel
* remove stray import
* followup dep change to moving the Equal[Name] instance
* add changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Name equal instance appears to be no longer used, but keeping anyway
* Support DAML-LF type synonyms in scala.
CHANGELOG_BEGIN
CHANGELOG_END
* dont create synonymns in GenerateSimpleDalf
* extend DAML-LF parser to support type synonyms
* test: expand type synonyms correctly
