The Engine does not verify key duplication when construction a
transaction, this task is currently let to the ledger implementation.
This PR provides an utility function to compute the duplicated keys
in a transaction.
CHANGELOG_BEGIN
CHANGELOG_END
* remove failedTransactions field from ScenarioLedger.RichTransaction
changelog_begin
changelog_end
* remove Blinding.checkAuthorizationAndBlind; fixup callers to use Blinding.blind when blindningInfo is required
* rename/relocate: ScenarioLedger.CommitError.FailedAuthorizations --> SError.DamlEFailedAuthorization
* fix types to demonstate that at most one FailedAuthorization is detected/reported
* address small review comments
* Speedy: group compiler parameters into a Config case class.
Speedy compiler is parametrized by several option flags. Those flags
are passed through different calls (in particular in the instance of
`CompiledPackages`.
This PR group all this configuration parameters into a case class to
pass the option in a cleaner way.
CHANGELOG_BEGIN
CHANGELOG_END
This PR implements a part of the proposal from #7093.
Here packages are validated in the participant node before to be sent to the ledger.
CHANGELOG_BEGIN
- [Ledger-API] participant node validate Dar before uploading to the ledger.
This may increase upload time significantly.
CHANGELOG_END
This is part of #7399.
This enable skipping packages validation in the engine when they are
comming from a trusted source.
CHANGELOG_BEGIN
CHANGELOG_END
changelog_begin
changelog_end
Adapt test for small error message change. An improvement! Previously an internal message was shown. Make test flexible enough to pass for old & new message.
Adapt expected output file. Only change is the contract-ids for active-contracts when the test ends.
fix small test bugs in testcase where authorization is wrong
temp disabled 3 tests; needs investigation; see TODO markers
temp adapt 1 test for change in error message
temp disable 1 test. needs invesigation
undo accidentally commited change to .bazelrc
add copyright header to new file
remove testcase (badActorCheck2) which is no longer expected behaviour, when authorization occurs during execution
address comments: be mre private & other tiny changes
appease scala formatter
improve expected error message in KeyNotVisibleStakeholders testcase
fix authorization issues and re-enable the 3 failing tests in EngineTest which now pass
fix auth issue and re-enable ledger-test-tool test: WronglyTypedContractIdIT
fix compatibility
re-enable test in KVUtilsTransactionSpec.scala
The migration script from #7344 ensures that there are no more values
in Sandbox Classic DB encoded with Value version < 6. Hence, we do
not need anymore a special EngineConfig to maintain backward
compatibility of Sandbox classic DB.
CHANGELOG_BEGIN
CHANGELOG_END
* LF: shortcup failure in SValue ordering.
Before this PR comparison of lists, maps, ans structural record had a
always linear complexity.
CHANGELOG_BEGIN
CHANGELOG_END
This reverts changed made in #3416, where the standard scala was
replace by ad-hoc Speedy equality for SValue. As Speedy equality
become more permisive (e.g. it does not check for type constructor of
records, variants, and enumrations or constructor of variants or
enumerations) this is safer to use standard scala eqality for case
class.
CHANGELOG_BEGIN
CHANGELOG_END
* DAML-LF spec: Fix two minor issues around structs
We make it explicit that all field names in a structural record need to
be distince. We also fix a small typo in the indices for the
`ExpStructCon` typing rule.
CHANGELOG_BEGIN
CHANGELOG_END
* Address comments regarding distinctness
CHANGELOG_BEGIN
CHANGELOG_END
This PR uses the new data structure introduced in #7220.
Additionally this fixes `svalue.Equality` and
`svalue.Ordering` which were considering <a: x, b: y>
different from <b:y, a:x>.
CHANGELOG_BEGIN
CHANGELOG_END
This PR uses the new data structure introduced in #7220.
Additionnally this fix `Value Equal instance` which was considering
<a: x, b: y> different from <b:y, a:x>.
CHANGELOG_BEGIN
CHANGELOG_END
This PR perform the following changes:
- reduce function size by spliting them out,
- rename "translate" function prefix to "compile",
- replace 'private' by 'private[this]',
- add 'inline' annotation to function used only once.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix trace statement handling in DAML Script service
This PR makes sure that we get both trace statements coming from the
ledger client as well as from the ledger server. In the script service
those are two different Speedy machines so we need to interleave
them. This is a bit messy so let me explain the steps (apologies for
not splitting it up but I think it’s easier to understand the
motivation when it is a single PR):
1. Abstract over the tracelog interface. The ringbuffer doesn’t makes
sense for our purposes and is annoying to clear.
2. Pass in the respective fields of `Machine` to `Conversions` instead
of the whole machine. This is both a simpler design (passing around
mutable objects scares me) and it allows us to assemble the fields
from different machines.
3. Initialize the ledger machine with a simple ArrayBuffer tracelog.
4. After `submit` and `submitMustFail` copy the tracelog from the
ledger to the client.
fixes#7280
changelog_begin
changelog_end
* Address review feedback
changelog_begin
changelog_end
* Only include stakeholder contracts in result of query
This fixes a bug in the script service. We need to filter out divulged
contracts since this should behave exactly like the ACS endpoint on
the ledger API.
changelog_begin
changelog_end
* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/scenario/ScenarioLedger.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
* fmt
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Refactor EnrichedTransaction
This PR prepares for work to interleave _authorization_ with _execution_, which has been an open issue for a long time: #132.
- Split computation of `BlindingInfo` and `FailedAuthorizations` into separate modules.
- Fixup callers: `Blinding.scala` and `ScenarioLedger.scala`, to do:
```
val failedAuthorizations =
AuthorizingTransaction.checkAuthFailures(authorization, tx)
val blindingInfo =
BlindingTransaction.calculateBlindingInfo(tx)
```
- Simplify code to remove `DontAuthorize` and `Authorization` types, which were only needed to restrict the combined _enrichment_ code from doing the authorization check when `blind`ing only is required. Now `Blinding.blind` calls just `calculateBlindingInfo`.
* address comments from Stefano
A `Struct[+X]` is a list of pair `(FieldName, X)` ordered by first
component.
We use this data-structure to represent TStruct LF type.
It will be used in upcomming PRs to sorted fields in Struct values.
CHANGELOG_BEGIN
CHANGELOG_END
* Normalize struct field order in the LF spec.
This commit changes the LF spec to ensure that struct field order is ignored whenever reasonable.
* Rename "type synonym resolution" to "type normalization", and change the rule for structs to also normalize the order of fields.
* Make the struct type "well-formed" only if the struct fields are sorted by field name. (This matches the way "well-formed" is only used for normalized types. E.g. type synonyms are not considered "well-formed" by the spec. This nomenclature should possibly be fixed in a separate PR.)
* Change the struct con expression type rule accordingly.
* Change the definition of a struct value. A struct value is now a struct expression where all the field expressions are values, and all the field names are sorted.
* Change the evaluation rule for struct expressions to also sort the fields by field name in the process.
As far as I know, this spec matches the behavior of the LF engine. I also tried to make the treatment of struct field order in the types match the treatment of struct field order in expressions (i.e. normalized structs always have sorted fields, but non-normalized structs do not).
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Update daml-lf/spec/daml-lf-1.rst
Co-authored-by: Remy <remy.haemmerle@daml.com>
This is primarily intended to improve error reporting in DAML Script,
in particular, https://github.com/digital-asset/daml/issues/7192 which
is a blocker for replacing scenarios by DAML Script.
However, I think it is also a very reasonable feature in general.
changelog_begin
changelog_end
lookupContract catches the exception which results in us continuing
and eventually running into a null-pointer exception.
This is another reason why we should enable the NonUnitStatements wart
but I’ll leave that for a separate PR (last I tried the scenario
service resulted in a ton of false positives).
fixes#7185
changelog_begin
changelog_end
This reverts commit 87a0ee9383.
CHANGELOG_BEGIN
WARNING remove the changelog entry from 87a0ee93 and starting by :
"[Sandbox] By default Sandbox rejects the development versions of ..."
CHANGELOG_END
This PR replaces a few `for (... to/until ...)` loops with `while`
loops which explicitly maintain their counter. In my benchmarks, this
has lead to a 3% performance improvement. Since the affected functions
are the ones used for applying arguments to functions, similar
improvements should be observable regardless of the specific benchmark.
CHANGELOG_BEGIN
CHANGELOG_END
* Add TO_TEXT_CONTRACT_ID primitive
This is the first part of #7114.
This PR
* Adds the primitive to the protobuf.
* Handles decoding and encoding in Haskell and Scala.
* Handles typechecking in Haskell and Scala.
* Handles speedy compilation and interpretation in Scala.
* Updates the specification.
This PR does not yet change the standard library to make use of this
primitive.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Avoid extra allocation
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* script service party management test-case
* script service: allocateParty and listKnownParties
Behave like sandbox on party hint. If a hint is given then allocate
exactly that party name and fail if it already exists. Otherwise
generate a fresh party name. For easier debugging the fresh name is
based on the display name.
changelog_begin
changelog_end
* Track parties that were not allocated explicitly
E.g. parties generated by `partyFromText`.
* Handle and test party already exists error
* fmt
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Originally those two options were set up by calling state mutating
method on a build engine.
In this PR, we move this two options to the recently introduced EngineConfig.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
* [Sandbox] By default Sandbox rejects the development versions of
DAML-LF and transaction format. One has to explicitly use the
command line option `--dev-mode` to allows those versions.
CHANGELOG_END
The methods moved take a single instance of the `Machine` class as an
argument, hence it makes no sense to have them in the companion object.
CHANGELOG_BEGIN
CHANGELOG_END
This PR simplifies the deprecated `getScenarioLedger` to take the
whole engine instead of `compiledPackages` (from the engine) and the
rest of the `outputTransactionVersions` (form the engine conffig).
This prepares for some changes for #5164, where more options for the
engine config will have to be used by the method. Also we will need to
be sure the `compiledPackages` following the option of the engine
config.
CHANGELOG_BEGIN
CHANGELOG_END
This PR addresses a TODO in the implementations of `foldl` and `foldr`.
The continuations for both primitives have to apply the step function to
the accumulcation and the current list item. So far, we've used the
`SEAppAtomicFun` AST node for this purpose. However, since the step
function and both arguments have already been evaluated to values, we
can also use the `enterApplication` function that has recently been
introduced together with our use of ANF.
Benchmarks: Applying `foldr (+) 0` (or `foldl (+) 0`) to the
pre-computed (and pre-allocated) list `[1..1_000_000]` has taken ca.
117s before this change and now takes only ca. 83ms. This is a speedup
of ca. 1.4x.
CHANGELOG_BEGIN
CHANGELOG_END
* add -Ywarn-unused to all scalac options
* remove some unused arguments
* remove some unused definitions
* remove some unused variable names
* suppress some unused variable names
* changeExtension doesn't use baseName
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* work around no plugins in scenario interpreter perf tests
* remove many more unused things
* remove more unused things, restore some used things
* remove more unused things, restore a couple signature mistakes
* missed import
* unused argument
* remove more unused loggingContexts
* some unused code in triggers
* some unused code in sandbox and kvutils
* some unused code in repl-service and daml-script
* some unused code in bindings-rxjava tests
* some unused code in triggers runner
* more comments on silent usages
- suggested by @cocreature; thanks
* fix missing reference in TestCommands
* more unused in triggers
* more unused in sandbox
* more unused in daml-script
* more unused in ledger-client tests
* more unused in triggers
* more unused in kvutils
* more unused in daml-script
* more unused in sandbox
* remove unused in ledger-api-test-tool
* suppress final special case for codegen unused warnings
.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
^
* one more unused in daml-script
* special scaladoc rules may need silencer, too
* unused in compatibility/sandbox-migration
* more commas, a different way to `find`
- suggested by @remyhaemmerle-da; thanks
This is a pre-step for ANF, which will allow ANF expression forms to call `executeMatchAlts` directly, rather than always from the `KMatch` continuation.
changelog_begin
changelog_end
This is another small PR split out from the ANF work.
Improve explore-dar to allow dar file to be selected on the command line, and so allows the json-parser example to be run like this:
bazel run daml-lf/interpreter/perf:explore-dar -- --base JsonParser pipeline --arg 10
As well as a speed test:
bazel run daml-lf/interpreter/perf:speed-json-parser
This example is a nice quick smoke test to check speedy execution is not broken.
changelog_begin
changelog_end
* DAML Engine: Speed up foldr
This PR replaces Speedy's current implementation of `foldr`, which
basically does nothing more than replace the builtin `foldr` with an
expression for its standard implementation in a functional language,
with a more loop-like implementation that takes advantage of the
structure of Speedy. There's additional complexity for the case when
the step function expects only one more argument before it performs
some computation. The exact issue and the solution are explained in a
comment in the code.
I've benchmarked the application of `foldr (+) 0` to the pre-computed
list `[1..100_000]`. With the old implementation of `foldl` this took
ca. 35ms, with the new implementation ca. 11ms. Further experiments
indicate that out of these times ca. 5ms are spent applying the
arguments to `(+)` and performing the addition. Thus, the time actually
spent in `foldr` amounts to 30ms and 6ms, respectively. This means
the new implementation of `foldr` is ca. 5x faster than the old
implementation when the step function takes at least two arguments.
For the case of a step function which expects only one argument before
performing computation, I've benchmarked the application of
`foldr (\_ -> identity) 0` to the pre-computed list `[1..100_00]`. The
measured times have dropped from ca. 42ms to ca. 19ms. Further
experiments that the time spent in `foldr` itself has dropped from
ca. 32ms to 9ms.
CHANGELOG_BEGIN
- [DAML Engine] The performance of `foldr` has been improved by more
than 4x.
CHANGELOG_END
CHANGELOG_BEGIN
CHANGELOG_END
* Add explanation on how extra coomplexity could have been avoided
CHANGELOG_BEGIN
CHANGELOG_END
* Extend the scenario service with DAML Script support
This adds most of the infrastructure for running DAML Script via the
scenario service which means it runs as part of DAML Studio and `daml
test`. This is hidden behind a feature flag so we can land this and
parallelize the remaining tasks. The main things that are missing are:
1. `createAndExerciseCmd` and `exerciseByKeyCmd`.
2. Party management needs some work and listing parties is
unsupported.
3. Time management
4. Potentially some better error handling (we need to go through
SResult and SError and see what is relevant for us).
Overall, it is already in a very usable state and there is a decent
range of tests.
closes#3688
changelog_begin
changelog_end
* Update compiler/damlc/daml-ide-core/src/Development/IDE/Core/Rules/Daml.hs
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Fix name for actor system and pool
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Stratify Speedy builtins into pure/effectful.
This is a preparatory change for the ANF translation, which can treat pure
builtins more efficiently.
- Parent: `SBuiltin`, (which are effectful).
- Child: `SBuiltinPure` (which are pure).
Effectful builtin functions may raise `SpeedyHungry` exceptions or change
machine state. Pure builtins can be treated specially because their evaluation
is immediate.
The interface to effectful builtins remains:
def execute(args: util.ArrayList[SValue], machine: Machine): Unit
For pure builtins the interface is:
def executePure(args: util.ArrayList[SValue]): SValue
changelog_begin
changelog_end
* insert trailing comma and newline in 17x execute method args-list
* fix spello
This PR replaces Speedy's current implementation of `foldl`, which
basically does nothing more than replace the builtine`foldl` with an
expression for its standard implementation in a functional language,
with a more loop-like implementation that takes advantage of the
structure of Speedy.
I've benchmarked the application of `foldl (+) 0` to the pre-computed
list `[1..100_000]`. With the old implementation of `foldl` this took
ca. 31ms, with the new implementation ca. 11ms. Further experiments
indicate that out of these times ca. 5ms are spent applying the
arguments to `(+)` and performing the addition. Thus, the time actually
spent in `foldl` amounts to 6ms and 26ms, respectively. This means
the new implementation of `foldl` is ca. 4.3x faster than the old
implementation.
CHANGELOG_BEGIN
- [DAML Engine] The performance of `foldl` has been improved by more
than 4x.
CHANGELOG_END
I've already grown very tired of almost always having to touch the
`Classify` class whenever I change anything about Speedy. Let's put an
end to this by using reflection instead of explicitly listing all
different AST nodes and continuation types.
The `PrettyLightweight` class has similar problems that can be fixed
using the same idea.
This change might slightly change this output but since this is an
experimentation tool, I deem this acceptable.
CHANGELOG_BEGIN
CHANGELOG_END
* FrontStack: Ensure that claimed invariant is maintained
The documentation of `FrontStack` claims that the head of the
`FQPrepend` is never empty but there's a way to violate this claimed
invariant.
This PR makes sure the invariant is maintained and then uses the
invariant to remove a redundant `else`-branch.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove initials from NOTE
CHANGELOG_BEGIN
CHANGELOG_END
* DAML Engine: Speed up updates of multiple fields in a record
Currently, updating multiple fields of a single record is not very
efficient. If you update, say, `n` fields, as in
```haskell
r with f_1 = ...; ...; f_n = ...
```
`n-1` intermediate records are created and almost immediately
discarded. Each creation of an intermediate record involves shallowly
cloning an array of size `m`, where `m` is the number of fields of the
record type. This does not only cost computation time but also a lot of
avoidable memory allocations. Overall, the cost of such an update is
`O(m*n)` in terms of time and memory.
This PR changes the DAML-LF interpreter such that multiple updates of
the same record are batched together. This avoids the creating of any
intermediate records but only creates the final record resulting from
the updates. This reduces the time complexity to `O(m+n)` and the
memory complexity to `O(m)`.
The code path for updating a single record field remains unchanged
(for now) and hence won't suffer from the additional complexity that
is required for the batching described above.
I've benchmarked this change by creating a single record with `m`
fields and updating all `m` fields in one single update expression,
for `m = 10` and `m = 20`. In both cases, the benchmarks showed a
speedup of ca. 2.5x.
This is part of #5766.
CHANGELOG_BEGIN
- [DAML Engine] Record update expressions of the form
`R with f_1 = E_1; ...; f_n = E_n` are now run as batch updates
in order to improve runtime performance and avoid unneccesary
memory allocations.
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
This is a spinoff from IDE support for DAML Script #6929. There I need
cannot quite use `run` from ScenarioRunner since I need to handle
results differently but I don’t want to replicate all the visibility
and authorization logic from `lookupContract` and `lookupKey`.
changelog_begin
changelog_end
* Use Bazel builtin pkg_tar rule
* Use @rules_pkg//:pkg.bzl%pkg_tar
The pkg_tar rule builtin to Bazel has been deprecated.
See https://docs.bazel.build/versions/master/be/pkg.html
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* daml-lf/parser: Add the ability to parse location annotations
Currently, the DAML-LF parser we use for testing in Scala land does not
understand location information and has in particular no way to produce
`ELocation` AST nodes. Having these nodes will be important in a test
for a feature I'm currently working on in order to demonstrate that my
Speedy AST transformation can actually look through location
information properly.
This PR adds a rule to the parser to allow for parsing location
information and producing `ELocation` nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* Address Nick's feedback
CHANGELOG_BEGIN
CHANGELOG_END
* Evaluation order for exercise & other updates.
This adds more DAML tests and error cases for, finishing off the update
interpretation for now. I believe this is roughly the limit of what we
can test directly in DAML, and the rest is up to DAML-LF testing.
For example, there's no way to test that the contract inactivity check
in EvUpdExercInactive happens before the evaluation of the controller
expression, since there is no way to generate an 'exercise' with actors
from DAML, as far as I could tell, and 'exercise_without_actor' always
evaluates the controller before passing it off to 'exercise'.
For the same reason, the EvUpdExercBadActor check seems impossible to
trigger from DAML itself. The closest is an authorization test that
happens during submit (which I added a test for, because it is kinda
relevant to evaluation order of scenarios, even though scenario
interpretation isn't really important to the spec).
So at the very least we should add LF tests for those two cases.
changelog_begin
changelog_end
* Apply Moritz\'s suggestions
* Add test with fetch of consumed contract
* add -Xlint:doc-detached
- reverts 1feae964e3 from #6798
* attach several scaladocs where they'll actually be included
* no changelog
* attach several more scaladocs where they'll actually be included
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Draft of new operational semantics.
* make errors deterministic, and be explicit about them
* Flesh out the op semantics for expressions and add tests.
* Nail down the semantics of create
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Add a reminder that u has to be a value
... in the update interpretation section.
changelog_begin
changelog_end
* add a test to track the ghc behavior on multi-lets
* add test for erasable type abstraction
* Add non-erasable type abstraction value test.
* move the ValExpTyAbsNat to 1.7-only value test
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Simplify inference of output transaction/value version.
As part of #5164, we simplify the way transaction and value versions
are inferred for the engine output.
The versioning of a transaction tr is done as follow:
* Let tvMin be the minimal transaction version allowed by the DAML
engine configuration
* Let tvMax be the maximal transaction version allowed by the DAML
engine configuration
* Let n₁, ..., nₘ be the nodes of tr.
* Let pkgᵢ be the package of the template associated to the node nᵢ
* Let lvᵢ be the maximal language version that pkgᵢ uses (directly or
through its dependencies)
* let tvᵢ be the maximal transaction version supported by all engines
that support both lvᵢ and tvMin
* Let vvᵢ be the maximal value version supported by all engines that
support tvᵢ.
* Let tv be the maximal transaction version between tv₁, ..., tvₘ.
* If tv is not greater than tvMax
+ then
- Version the values of each node nᵢ according vvᵢ
- Version tr according tv
+ fail otherwise
CHANGELOG_BEGIN
CHANGELOG_END
* set many extra scalac -Xlint options for all Scala projects
CHANGELOG_BEGIN
CHANGELOG_END
* move NoCopy to its own file
package.scala:18: warning: it is not recommended to define classes/objects inside of package objects.
If possible, define trait NoCopy in package data instead.
trait NoCopy {
^
* move more traits, classes, and objects to proper packages
- note that `package` is itself a scoping construct, so if your reason
is the apparent aesthetic of placing a bunch of things in one `package
object`, that is easily remedied by deleting the `object` keyword
* fix some type-parameter-shadow warnings
- I'm generally in favor of sensible name-shadowing, following the
"deliberately hide variables that should not be accessed here" school
of thought. But I think type name shadowing isn't quite as valuable
and more likely to confuse than general variable shadowing, so have
experimentally linted it out.
Example warning:
EventsTableFlatEventsRangeQueries.scala:11: warning: type parameter
Offset defined in trait EventsTableFlatEventsRangeQueries shadows class
Offset defined in package v1. You may want to rename your type
parameter, or possibly remove it.
private[events] sealed trait EventsTableFlatEventsRangeQueries[Offset] {
^
* fix more package-object-classes warnings
* fix an inaccessible warning
ContractsService.scala:197: warning: method searchDb in class ContractsService references private class ContractsFetch.
Classes which cannot access ContractsFetch may be unable to override searchDb.
def searchDb(dao: dbbackend.ContractDao, fetch: ContractsFetch)(
^
* enable -Xlint:infer-any
- continuing the saga of #6116, #6132
* enable -explaintypes for more detailed type errors
* missed header for NoCopy; probably should have left it in the package file
* misspelling in comment
* revert -Xlint:doc-detached
- there are a lot of these fixes, and they are noisy, so shifting to a
separate PR
- thanks to @leo-da for pointing out
According transaction specification the field `value_version` "is
optional; if defined, it must be a version ofthe value
specification, and `template_id` shall be consumed according to
that version. Otherwise, it is assumed to be version 1."
We take advantage that the encoding of identifier in value does not
change since version "1" to unset the field `value_version`, hence
using the default interpretation.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
[Engine] - Change the callback for contract key from `GlobalKey => Option[ContractId]` to `GlobalKeyWithMaintainers => Option[ContractId]`
CHANGELOG_END
We add a method to CompiledPackages that returns the latest language version that package uses directly or indirectly through is dependencies.
This will be used in the context of #5164
CHANGELOG_BEGIN
CHANGELOG_END
SubmittedTransaction and CommittedTransaction are moved from com.daml.lf.transaction.Transaction to
com.daml.lf.transaction
This helps intelliJ type inference.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
Default new Engine constructor to Engine.StableConfig so it does not
need to be overridden unless you specifically want to run in
Engine.DevConfig mode
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* include DefTemplate's key types under the Ty type variable
* note that contract key types are included by folding over DefTemplates' Tys
- the topo sort from --root now correctly includes contract keys
* test that contract key dependencies get included in the Scala codegen plan
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* Revert ANF changes and add a testcase for evaluation order
After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.
Therefore, this PR reverts the following commits:
1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee
Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably
1. https://github.com/digital-asset/daml/pull/6576 which was easy to
resolve and the change to return SEValue instead of SExpr is still
nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
some changes since the constructors changed. If you want to review
those changes in detail (they are pretty straightforward so not too
important), it’s probably easiest to check out this PR and run
```
git diff 2cd2a8f2a8
daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
```
to see the diff to the parent commit of the first commit that
introduced ANF.
changelog_begin
changelog_end
* replace traverseU and sequenceU with traverse and sequence
- with -Ypartial-unification on, the extra Unapply typeclass lookup is
unnecessary
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* limit imports; we only need *> and void
So that the resulting value is returned from the `execute` function, instead of being assigned to the `returnValue` machine state.
Rename the parent class from `SBuiltinMaybeHungry` to `SBuiltinEffect`, and have this be the parent of a few more subclasses which cannot meat the _pure_ interface.
Declare method `executeEffect` of the parent class `SBuiltinEffect` which has the same type as the original `execute`.
changelog_begin
changelog_end
Ask @nickchapman-da for details on what this transform does and why it's
better. I have no clue.
What I do understand is that current master has a stack-consuming
behaviour that just so happens to not reach the 1mb default stack size
on the tests we run, and that this presumably better version of a let
transform had been left out from the original ANF PR because it was
pushing the stack consumption over that 1mb limit.
Besides reintroducing the "better" let version, this PR applies an
almost full CPS transform to the set of mutually recursive functions
that grow the stack. Combined with the trampoline, this should mean the
ANF transform itself is done in (almost) constant stack space.
The one exception to that is the `flattenAlts` function, which has been
left as a plain imperative loop. This does mean that we consume stack
linearly with the nesting level of alts, so conceivably this could still
blow the stack. There is no fundamental reason that makes this hard, it
just seemed unnecessary for the stack consumption and potentially
damaging to performance (changing a native Java loop on an array to a
recursive list consumption).
To verify the stack-consuming behaviour on master, one can apply this
patch:
```patch
diff --git a/ledger/sandbox/BUILD.bazel b/ledger/sandbox/BUILD.bazel
index a717e66fb..4c953b3ea 100644
--- a/ledger/sandbox/BUILD.bazel
+++ b/ledger/sandbox/BUILD.bazel
@@ -124,6 +124,7 @@ da_scala_library(
da_scala_binary(
name = "sandbox-binary",
+ jvm_flags = ["-Xss256k"],
main_class = "com.daml.platform.sandbox.SandboxMain",
resources = ["src/main/resources/logback.xml"],
visibility = ["//visibility:public"],
```
then run
```
bazel test -t- //ledger/sandbox:conformance-test-contract-id-seeding-memory
```
The same process can be repeated on this branch to verify that, at
least, this now runs under 256kb of stack for all our tests. I don't
know of a good way to get a stronger guarantee on stack allocations.
While my primary motivation here is correctness (and termination, I
guess), this does seem to yield a modest performance improvement. On a
test machine, I got the following results as confidence intervals:
* `master` (b4915a4bd7): 54.00, 54.56
* this branch: 52.54, 53.11
I've only ran the benchmark once on each and don't have enough
experience with it to know how reliable those numbers are. I'll try to
run some more.
CHANGELOG_BEGIN
CHANGELOG_END
small clean up in the testing library TransactionBuilder, bsaically add a Submitted and a Committed version for empty transaction constant and build method.
plus a bit of cleaning in tests using TransactionBuilder
CHANGELOG_BEGIN
CHANGELOG_END
Since we have only absolute contract ids, there is no more distinction
between local and global disclosure/divulgence. Let's please remove the
`global` prefix since it causes confusion (at least for me it did).
CHANGELOG_BEGIN
CHANGELOG_END
The dead code was hanging on by a slender thread. This change cuts that thread.
For over-apps, we now push a new continuation type KOverApp, instead of KArg.
KOverApp contains args of type SExprAtomic, instead of SExpr, and so its
execute() method may use enterApplication, instead of executeApplication.
This allowing lots of code to be removed:
- the continuation types: KArg, KFun, KBuiltin and KPap.
- defs: evaluateArguments and executeApplication.
changelog_begin
changelog_end
Currently, the scenario service ships the disclosure information of a
transaction twice. One of the two copies is completely unused. This PR
simply deletes it. I hope this reduces confusion in the future.
CHANGELOG_BEGIN
CHANGELOG_END
* Distinguish witnesses and divulgences in scenario table view
Currently, we display a "D" (for "Disclosure/Divulgence") in the table
view of the scenario view when a party can see a contract because they
have learnt about it via disclosure.
This change further refines this visibily indicators for disclosure:
We display a "W" for "Witness" (i.e. the party has witnessed the
creation of the contract) and "D" for "Divulgence".
Help for the reviewer: In the scenario service server, witnessing is
called explicit disclosure whereas divulgence is called implicit
disclosure.
CHANGELOG_BEGIN
* [DAML Studio]
When displaying scenario results in table view in DAML Studio, we now
indicate _why_ a party knows about the existence of a contract:
- `S` means the party is a signatory.
- `O` means the party is an observer.
- `W` means the party has witnessed the creation of the contract.
- `D` means the party has learned about the contract via divulgence.
CHANGELOG_END
* Address @cocreature's feedback
I was trying to track down what looks like a new source of flakiness in
the ANF work (#6440), which manifested in a stack overflow on the
recursion of the `go` function in speedy.Compiler. I spent a bit of time
refactoring this to be functional in order to then trampoline it, only
to realize afterwards that I had worked on `freeVars/go` whereas the one
blowing up the stack was `validate/go`.
Obviously, it did nothing to reduce the rate of flakiness on the stack
overflow issue I was trying to address, as it's a separate function.
So there isn't really a good reason for this change, except I had done
it and I do believe the code looks marginally nicer, so might as well
submit it.
CHANGELOG_BEGIN
CHANGELOG_END
* ANF transformation in Speedy.
The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.
changelog_begin
changelog_end
* improve comment
* inline functions relocateA/L
* remove comment about scalafmt
* remove commented out alterative def for transformLet1
* improve code by adding incr methods to DepthA/E
* remove (n == 0) special case in trackBindings
* clarify comment further
* improve validate/go to not consume stack for deeply right-nested let-expressions
* address comments from Remy: be private; use final case case; etc
* rename to unsafeCompilationPipeline
* add back some trailing commas
* remove commented-out debug line
* improve comment
* remove dev/debug code in compilationPipeline
* remove commented out code in SEAppGeneral.execute
* undo unrelated code improvement in SValue.scala
* fix compile. object Anf cannot be private
* Measure time of interpretation and store it in CommandExecutionResult.
* Added parameters for passing in interpretation time for WriteService and LedgerWriter.
* Code tidying.
* Added CommitMetadata parameter to LedgerWriter.
* Thread through interpretation time. Aggregate interpretation times for a batch.
* Take max of interpretation times.
* Calculate interpretation time in nanos.
* Moved CommitMetadata into separate file.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Delegate deprecated methods to method with new signature.
* Code tidying.
* Suppress deprecation warnings.
* Made interpretation cost optional in CommitMetadata.
* Test that we populate interpretation time.
* Code tidying.
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Code tidying.
* Some more tests. Do not return interpretation cost for a batch if it only contains non-transaction submissions.
* Reformatted.
* Reformatted.
* Named arbitrary interpretation cost.
* Reverted changes for BatchingLedgerWriter.
* Always drop commit metadata for batches and don't report interpretation cost.
* More specific expectations.
* Include authorization check and blinding in interpretation time.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
So far, these two flags only worked in sandbox classic because I wasn't
aware the code path consuming the options is not shared between both
implementation. Now, sandbox next is on par with sandbox classic.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Add a command line flag to disable DAML stack traces
The sandbox now accepts a `--stack-traces no` flag which will turn off
the location tracking in DAML Engine required to produce stack traces
for failing DAML code.
Benchmarks suggest that DAML Engine spends about 10% of its time with
tracking locations. Thus, this flag will give us roughly a 1.1x
speedup when stack traces are not needed.
This flag is still hidden because we would like to validate its
usefulness before we commit to supporting it.
CHANGELOG_BEGIN
CHANGELOG_END
* Make it more obvious where we're overriding methods
CHANGELOG_BEGIN
CHANGELOG_END
* Improve help text
* Address https://github.com/digital-asset/daml/pull/6507#discussion_r446113575
* drop blindinginfo.proto
changelog_begin
changelog_end
* drop BlindingCoder
* Remove blindinginfo Protobuf definition JAR
changelog_begin
[DAML-LF] The blindinginfo Protobuf definition JAR, which was previously unused, has been pulled from the artifacts released on Maven
changelog_end
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
When trying to understand how disclosure/divulgence information is
computed for the scneario service, I stumbled on some dead code:
* The `EnrichState.divulgeContracts` function is never used. Let's
remove it.
* The `EnrichState.localDivulgendes` field is initialized with an empty
map and never touched again. Thus, it's always going to stay an empty
map. This means that the
`EnrichedTransaction.localImplicitDisclosure` is also always going to
be an empty map. Both fields can hence be removed.
I'm not sure if the `BlindingInfo.localDivulgence` field can be removed
as well. In my understanding, this data structure could be loaded from
transactions serialized with the field being non-empty in the past.
Thus, I've refrained from removing the field and set it to an empty map
when constructing a `BlindingInfo` from an `EnrichedTransaction`.
CHANGELOG_BEGIN
CHANGELOG_END
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
* Fix contains check in ConcurrentCompiledPackages
`contains` searches for a _value_ not a _key_. We have a package id
here in both cases which is clearly a key.
I am not exactly clear on what exactly happens if you hit this bug. I
believe it’s just a performance issue so probably hard to write a test
for.
I did take a brief look at whether we can make this
typesafe (`contains` accept an `Object` which is how this typechecks)
and apparently calling `asScala` and then using
`scala.collection.concurrent.Map` should work but I don’t know if this
has performance implications or if there is another reason why we
didn’t do this. Happy to make the change if someone tells me this is
the right thing to do.
changelog_begin
changelog_end
* Switch to Scala’s concurrent map which doesn’t pretend types are bad
changelog_begin
changelog_end
At the moment, JMH seems happy to just swallow exceptions and consider
the benchmark done, which makes it produce inaccurate speed results and
lets errors slip through to master. This makes unexpected errors in the
benchmark a hard stop.
This is not a complete solution: ideally there would be a way to just
tell jmh to abort on uncaught exceptions. However, I don't seem to be
able to find any relevant documentation on how to do that.
CHANGELOG_BEGIN
CHANGELOG_END
During some refactoring we forgot to save the initial expression to
evaluate for the machine during benchmarking. This PR fixes the issue.
It also make the error messages a bit more descriptive so that we can
actually debug this.
A test to make sure issues like this one don't get through CI again is
worked on by @gary-verhaegen-da in a separate PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Consistently display stakeholders for key visibility errors
fixes#6404
As pointed out by Bernhard in #6404, the previous behavior was pretty
weird. If the committer was only a divulgee, we only displayed
stakeholders. If the committer was neither a stakeholder nor a
divulgee, we displayed stakeholders + parties the contract has been
divulged to. Given that only stakeholders can do lookups it makes much
more sense to display them consistently which is what this PR
achieves. I’ve also renamed “disclosed to” to “stakeholders” to make
it very explicit what is shown there.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* fmt
changelog_begin
changelog_end
* lalala
changelog_begin
changelog_end
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
fixes#6403
I am not entirely sure why I thought that using `missingWith` makes
sense here but it clearly doesn’t make sense and resulted in a pretty
bad bug where a transaction both succeeded via `submit` as well as
failed via `submitMustFail` which is clearly the wrong thing to do.
This PR fixes this issue and introduces a `notVisibleWith` function
that does the right thing. I’ve also added some comments and an extra
assertion to clarify things a bit.
changelog_begin
changelog_end
In this PR we cleanup the constructor for the speedy Machine.
* We remove the `case` keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
+ one generic builder `Machine.apply`,
+ scenario specific builder,
CHANGELOG_BEGIN
CHANGELOG_END
* use Profile.Label newtype for typechecked union instead of AnyRef
- includes port of some of interpreter
- demonstrating its efficacy is the compiler error in this commit:
daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala:267: error: type mismatch;
found : com.daml.lf.speedy.SExpr.SEBuiltinRecursiveDefinition
required: com.daml.lf.speedy.Profile.Label
(which expands to) com.daml.lf.speedy.Profile.LabelModule.Module.T
withLabel(ref, ref)
^
What was likely intended was to write `ref.ref` here; that is the assumption
the `Event#label` rendering function makes, anyway. Now, we type-check that
the labelling matches the renderer.
* let Profile make arbitrary Labels
* fix null and missing `.ref` calls for labelling
* one more null => LabelUnset
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* implicitNotFound message never used
* ritual offering of the dot and parens
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* LF: rename library transaction-scalacheck to transaction-test-lib
CHANGELOG_BEGIN
CHANGELOG_END
* move files in com/daml
* missing change in release/artifacts.yaml
* remove 'com/dam' from the path
We split the object com.daml.lf.types.ledger in three:
- one part in `com.daml.lf.ledger.` (in //daml-lf/transaction) for the part relative to EventId (shares between scenario service and sandbox)
- one part in `com.daml.lf.ledger.` (in //daml-lf/interpreter) for the part common to Blinding and Scenario
- one part in `com.daml.lf.scenario.` (in //daml-lf/interpreter) for the part specific to Scenario
fixes#6260
CHANGELOG_BEGIN
CHANGELOG_END
This PR adds some DAML speed benchmarks which focus on the computational aspects of DAML.
The first benchmark is `nfib`. The speed is reported in nfibs/micro-second.
The second benchmark is `json-parser`. We have a short pipeline: JSON AST is constructed
to represent an arithmetic expression. The AST is converted to its string representation.
The JSON string is then parsed back to AST using the JSON parser (which is defined using
parser combinators defined in the benchmark code). Finally the arithmetic expression
embedded in the JSON AST is evaluated. We report the speed in k-chars/second.
The speed tests are designed to be quick and easy to run. Both tests scale exponentially
in their integer argument, and so are easy to tune so each iteration takes about half a
second. The are run like this:
```
bazel run daml-lf/interpreter/perf:nfib
bazel run daml-lf/interpreter/perf:speed-json-parser
```
For interest, the speeds I see on my dev machine are:
- nfib: 1.35 nfibs/us
- json-parser: 27 k/s
changelog_begin
changelog_end
This PR fixes a bug where the profiler wrote open events for functions
_before_ evaluating their arguments. However, in a call-by-value
language such as DAML, arguments are evaluated before entering a
function and the profiler should reflect that.
This PR also adds a regression test.
CHANGELOG_BEGIN
CHANGELOG_END
In order to make the flamegraphs easier to consume by humans, we change
the profiler output in the following ways:
1. Don't print package ids anymore. They are not particularly useful
but cause a lot of noise.
2. Remove a few useless angle bracket and move the printed names of
DAML-LF closer to their surface level names.
3. Unmangle identifiers on a best effort basis.
4. Give the profiles shorter names such that they don't occupy the
whole screen and leave some space for the navigation buttons of the
Speedscope UI.
CHANGELOG_BEGIN
CHANGELOG_END
* Optimize the execution of Saturated Builtin Applications in Speedy.
We special case applications where the expression in function-position is a builtin operator, and the number of arguments matches the arity of the builtin. The special-case detection is done at compile time, and allows for more efficient runtime execution, specifically:
- We don't need to construct an `SPAP` value, only to immediately deconstruct/enter it.
- We don't need to do arity checking at runtime, with special case handling for _partial-_ and _over-_ applications.
The change gives about 3% speedup.
changelog_begin
changelog_end
* improve doc comments & make class names more descriptive
* share code for evaluating arguments
* improve name: SEAppSaturatedBuiltinFun
* optimize over-applied builtin function applications
* fix bug in the refactoring which introduced evaluateArguments
