Commit Graph

135 Commits

Author SHA1 Message Date
Remy
58b1c4e771
Speedy: Refactor contract Id/Key callback (#10033)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-22 16:34:28 +02:00
Moritz Kiefer
e4c1c58317
Run scenarios in off-ledger machine (#10070)
* Run scenarios in off-ledger machine

This PR builds on the previous PR that split scenario execution in two
different speedy machines and now actually makes the machine that runs
scenarios run in off-ledger mode just like we handle Daml Script.

This required a bunch of refactoring to make it nice so apologies for
the slightly large PR. Hopefully it’s still relatively easy to follow
and luckily it deletes more code than it adds.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* review comments

changelog_begin
changelog_end

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-06-22 12:11:00 +02:00
Moritz Kiefer
fb757d8ed9
Handle visibility outside of speedy (#10056)
It makes no sense to pass NotVisible to Speedy especially since that
is not how visibility is handled for the Engine. Also it lets us
delete code and I like deleted code.

changelog_begin
changelog_end
2021-06-21 17:31:37 +02:00
Moritz Kiefer
febca5d62d
Use ScenarioRunner.submit in Daml Script (#10053)
* Use ScenarioRunner.submit in Daml Script

changelog_begin
changelog_end

* privatize ledger variable

changelog_begin
changelog_end

* drop space

changelog_begin
changelog_end
2021-06-17 17:12:45 +00:00
Moritz Kiefer
2b915e93da
Split scenario & ledger execution (#10039)
* Split scenario & ledger execution

This PR by no means aims to solve everything we can do here. It is
rather the minimal change that I could get to work that provides us
with one Speedy machine for scenario execution (which is still an
on-ledger machine to avoid having to change the callsites to much in
this PR) and one speedy machine per submission.

There is tons of cleanup we can do afterwards but this should
hopefully set the right foundations.

changelog_begin
changelog_end
2021-06-17 12:45:35 +02:00
Moritz Kiefer
836a82a6b6
Add proper error handling for missing contract keys (#9972)
* Add proper error handling for missing contract keys

These are clearly not internal errors so we should not be calling
`crash` here. Canton in fact already started string matching on the
crash message which is definitely not what we want.

changelog_begin
changelog_end

* backwards compat

changelog_begin
changelog_end

* fix tests

changelog_begin
changelog_end
2021-06-14 09:12:30 +00:00
Remy
7bc925e4d2
LF: Factorize the logic for AST lookup (#9871)
Factorize the logic for AST lookup in
 - compiler 
 - type checker 
 - preprocessor
 - script triggers 
 - some tests

CHANGELOG_BEGIN
CHANGELOG_END
2021-06-03 13:32:56 +02:00
Moritz Kiefer
4e1c5fedb3
Check visibility for by-key operation of local contracts (#9470)
* Check visibility for by-key operation of local contracts

fixes #9454

I tried out two approaches for this:

1. The one here where we add a new callback. This has the advantage
   that the engine remains oblivious to visibility checks. They are all
   done outside and the engine doesn’t even know about the reading
   parties.
2. Make the engine aware of the reading parties. A start of that is in
   #9458.

Both work in principle but I ended up going for 1 in the end. Doing
half of the visibility checks outside the engine and half inside just
seems worse than the current state.

changelog_begin

- [Daml Engine] Fix a bug where it was possible to
  fetch/lookup/exercise a local contract by key even if the reading parties
  are not stakeholders. See #9454 for details.

changelog_end

* Disable new test on Canton

changelog_begin
changelog_end

* Exclude from compat tests

changelog_begin
changelog_end

* s/LocalLookup/LocalFetch/

changelog_begin
changelog_end

* Address review

changelog_begin
changelog_end
2021-04-22 21:39:12 +02:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Moritz Kiefer
0af4f522c7
Port the rest //daml-lf/... to Scala 2.13 (#8436)
* Port the rest //daml-lf/... to Scala 2.13

Draw the rest of the owl

changelog_begin
changelog_end

* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-01-08 10:38:17 +01:00
Moritz Kiefer
424faa923a
Port damlc dependencies to Scala 2.13 (#8423)
* Port damlc dependencies to Scala 2.13

I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).

So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).

If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.

changelog_begin
changelog_end

* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* fixup lawlesstraversal

changelog_begin
changelog_end

* less iterator more view

changelog_begin
changelog_end

* document safety of unsafeWrapArray

changelog_begin
changelog_end

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-01-08 07:22:38 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Remy
44c5b8a777
LF: remove version from value nested in GenNode. (#8217)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:12:14 +01:00
Moritz Kiefer
5c077a4550
Add multi-party submissions to DAML Script (#8240)
* Add multi-party submissions to DAML Script

changelog_begin

- [DAMl Script] Add early-access submitMulti and submitMultiMustFail
  for multi-party submissions. This is only supported in DAML Studio at
  the moment.

changelog_end

* Fix daml-lf prettyprinter

changelog_begin
changelog_end

* Cleanup visibleIn

changelog_begin
changelog_end

* Fix JsonApiIt

changelog_begin
changelog_end

* s/Set()/Set.empty

changelog_begin
changelog_end

* Cleanup token check

changelog_begin
changelog_end

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Document ParticipantView

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-12-11 12:45:32 +01:00
Moritz Kiefer
7e05dc7932
Upgrade rules-scala and scalatest (#8187)
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.

This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.

Apologies for the giant PR, I don’t see a way to keep it smaller.

changelog_begin
changelog_end
2020-12-08 06:59:23 +01:00
Samir Talwar
9976b4cd50
Bazel: Factor out logic around Protobuf JARs. [KVL-714] (#8084)
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.

* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.

* Bazel: Factor out Java/Scala protobuf class generation into a helper.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/archive: Use `proto_jars`.

* Bazel: Remove the visibility modifier from `proto_jars`.

It's too confusing. Just make everything public.

* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.

* Bazel: Add comments to the various parts of `proto_jars`.

* daml-assistant: Do unpleasant things with `location` in Bazel.
2020-11-27 08:34:53 +00:00
Remy
21c1c951f1
LF: remove control of input/output value/transaction versions. (#7858)
This is made obsolete by #7788.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-02 20:02:53 +01:00
Moritz Kiefer
cd4f085e92
Support multiple parties in scenario’s ParticipantView (#7666)
As part of multi-party read/write on command submissions we also need
to be able to support multi-party queries in DAML Script. Since the
queries in DAML Studio go via ParticipantView, this PR extends this as
a prerequisite to making use of this in DAML Script.

This does not yet relax any restrictions on command submissions. Those
still require a single committer.

For consistency with the surrounding code, I went for a simple Set
rather than a non-empty Set.

changelog_begin
changelog_end
2020-10-14 12:51:02 +02:00
Moritz Kiefer
00b80b8ea3
Separate off-ledger ond on-ledger speedy (#7501)
* Separate off-ledger ond on-ledger speedy

changelog_begin
changelog_end

* Add OnLedgerBuiltin abstract class

changelog_begin
changelog_end

* fix typo

changelog_begin
changelog_end

* Fix borked rebase

changelog_begin
changelog_end

* Remove unscoped withOnLedger

changelog_begin
changelog_end
2020-09-29 10:05:45 +00:00
Remy
76ef4c861a
Speedy: group compiler parameters into a Config case class. (#7438)
* Speedy: group compiler parameters into a Config case class.

Speedy compiler is parametrized by several option flags.  Those flags
are passed through different calls (in particular in the instance of
`CompiledPackages`.

This PR group all this configuration parameters into a case class to
pass the option in a cleaner way.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-18 19:32:26 +02:00
Moritz Kiefer
255832cefc
Fix failed contract lookups in scenario runner (#7190)
lookupContract catches the exception which results in us continuing
and eventually running into a null-pointer exception.

This is another reason why we should enable the NonUnitStatements wart
but I’ll leave that for a separate PR (last I tried the scenario
service resulted in a ton of false positives).

fixes #7185

changelog_begin
changelog_end
2020-08-20 09:49:03 +02:00
Remy
66ae38da6c
LF: control of input value version in the engine (#6828)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-13 21:15:52 +02:00
Remy
6d02e20764
Scenarios: simplify getScenarioLedger method. (#7125)
This PR simplifies the deprecated `getScenarioLedger` to take the
whole engine instead of `compiledPackages` (from the engine) and the
rest of the `outputTransactionVersions` (form the engine conffig).

This prepares for some changes for #5164, where more options for the
engine config will have to be used by the method. Also we will need to
be sure the `compiledPackages` following the option of the engine
config.

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-13 16:41:46 +02:00
Stephen Compall
96624a7677
use -Ywarn-unused for all Scala code (#6907)
* add -Ywarn-unused to all scalac options

* remove some unused arguments

* remove some unused definitions

* remove some unused variable names

* suppress some unused variable names

* changeExtension doesn't use baseName

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* work around no plugins in scenario interpreter perf tests

* remove many more unused things

* remove more unused things, restore some used things

* remove more unused things, restore a couple signature mistakes

* missed import

* unused argument

* remove more unused loggingContexts

* some unused code in triggers

* some unused code in sandbox and kvutils

* some unused code in repl-service and daml-script

* some unused code in bindings-rxjava tests

* some unused code in triggers runner

* more comments on silent usages

- suggested by @cocreature; thanks

* fix missing reference in TestCommands

* more unused in triggers

* more unused in sandbox

* more unused in daml-script

* more unused in ledger-client tests

* more unused in triggers

* more unused in kvutils

* more unused in daml-script

* more unused in sandbox

* remove unused in ledger-api-test-tool

* suppress final special case for codegen unused warnings

.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
      implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                         ^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
      implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                               ^

* one more unused in daml-script

* special scaladoc rules may need silencer, too

* unused in compatibility/sandbox-migration

* more commas, a different way to `find`

- suggested by @remyhaemmerle-da; thanks
2020-08-07 13:16:09 -04:00
Remy
ae7e92a6c4
LF: adapt TransactionBuilder to the new version inference (#6854)
This PR changes TransactionBuilder according the new version inference
algorithm implemented in #6756

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-03 19:05:30 +02:00
Moritz Kiefer
83031f3edf
Mark contract lookups as private[lf] (#6938)
DAML Script is in lf so that is good enough and nothing else should
access this.

changelog_begin
changelog_end
2020-07-30 17:28:23 +00:00
Moritz Kiefer
13add95a5e
Expose lookupContract and lookupKey methods from ScenarioRunner (#6931)
This is a spinoff from IDE support for DAML Script #6929. There I need
cannot quite use `run` from ScenarioRunner since I need to handle
results differently but I don’t want to replicate all the visibility
and authorization logic from `lookupContract` and `lookupKey`.

changelog_begin
changelog_end
2020-07-30 18:15:48 +02:00
Remy
ce3c3c89a4
LF: redesign engine configuration (#6763)
This PR redesigns the engine Configuration as described in #5164. 

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-22 12:17:07 +02:00
Remy
5468e4d160
DAML-LF: Simplify inference of output transaction/value version. (#6756)
* LF: Simplify inference of output transaction/value version.

As part of #5164, we simplify the way transaction and value versions
are inferred for the engine output.

The versioning of a transaction tr is done as follow:

* Let tvMin be the minimal transaction version allowed by the DAML
  engine configuration
* Let tvMax be the maximal transaction version allowed by the DAML
  engine configuration
* Let n₁, ..., nₘ be the nodes of tr.
* Let pkgᵢ be the package of the template associated to the node nᵢ
* Let lvᵢ be the maximal language version that pkgᵢ uses (directly or
  through its dependencies)
* let tvᵢ be the maximal transaction version supported by all engines
  that support both lvᵢ and tvMin
* Let vvᵢ be the maximal value version supported by all engines that
  support tvᵢ.
* Let tv be the maximal transaction version between tv₁, ..., tvₘ.
* If tv is not greater than tvMax
  + then
    - Version the values of each node nᵢ according vvᵢ
    - Version tr according tv
  + fail otherwise

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-21 17:07:11 +02:00
Remy
a629959c09
LF: Move GlobalKey and GlobalKeyWithMaintainers outside Node (#6795)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-21 09:08:36 +02:00
Remy
5f4dd39b5d
Engine: add maintainers to contract key call back (#6781)
CHANGELOG_BEGIN
[Engine] - Change the callback for contract key from `GlobalKey => Option[ContractId]` to  `GlobalKeyWithMaintainers => Option[ContractId]`
CHANGELOG_END
2020-07-20 17:42:06 +02:00
Remy
15da222939
DAML-LF: move some type definitions out of Transaction object (#6739)
SubmittedTransaction and CommittedTransaction are moved from com.daml.lf.transaction.Transaction to
com.daml.lf.transaction

This helps intelliJ type inference.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-16 09:13:30 +02:00
Moritz Kiefer
02c59d4f2a
Revert ANF changes and add a testcase for evaluation order (#6645)
* Revert ANF changes and add a testcase for evaluation order

After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.

Therefore, this PR reverts the following commits:

1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee

Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably

1. https://github.com/digital-asset/daml/pull/6576 which was easy to
   resolve and the change to return SEValue instead of SExpr is still
   nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
   some changes since the constructors changed. If you want to review
   those changes in detail (they are pretty straightforward so not too
   important), it’s probably easiest to check out this PR and run
   ```
   git diff 2cd2a8f2a8
   daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
   ```
   to see the diff to the parent commit of the first commit that
   introduced ANF.

changelog_begin
changelog_end
2020-07-08 12:40:12 +02:00
Remy
1480168604
Speedy: make thing a bit more private (#6579)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-02 18:19:03 +02:00
nickchapman-da
353d0da6f7
Speedy ANF (#6440)
* ANF transformation in Speedy.

The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.

changelog_begin
changelog_end

* improve comment

* inline functions relocateA/L

* remove comment about scalafmt

* remove commented out alterative def for transformLet1

* improve code by adding incr methods to DepthA/E

* remove (n == 0) special case in trackBindings

* clarify comment further

* improve validate/go to not consume stack for deeply right-nested let-expressions

* address comments from Remy: be private; use final case case; etc

* rename to unsafeCompilationPipeline

* add back some trailing commas

* remove commented-out debug line

* improve comment

* remove dev/debug code in compilationPipeline

* remove commented out code in SEAppGeneral.execute

* undo unrelated code improvement in SValue.scala

* fix compile. object Anf cannot be private
2020-06-30 20:52:44 +00:00
Remy
6d27825acb
Engine: introduce config to control value/transaction output version (#6428)
* Engine: introduce configuration to control value/transaction output version

This advances the state of #5164

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-26 13:38:38 +02:00
Stephen Compall
a51d0db8ff
set scalac -Xsource:2.13 -Ypartial-unification globally (#6469)
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts

* add now-referenced scalaz-core where needed

* work around bad type signatures in scalatest Aggregating, Containing

* unused Any suppression

* work around bad partial-unification wrought by type alias

* remove unused Conversions import

- not required in 4f68cfc480 either, so unsure how it's survived this long

* work around Future.traverse; remove unused show import

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* remove unused bounds

* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed

* longer comment on what the options do

- suggested by @stefanobaghino-da; thanks

* forget Future.traverse, just use scalaz, it knows how to do this
2020-06-24 16:51:24 -04:00
Gary Verhaegen
d48efd35c7
exit on failed benchmark (#6448)
At the moment, JMH seems happy to just swallow exceptions and consider
the benchmark done, which makes it produce inaccurate speed results and
lets errors slip through to master. This makes unexpected errors in the
benchmark a hard stop.

This is not a complete solution: ideally there would be a way to just
tell jmh to abort on uncaught exceptions. However, I don't seem to be
able to find any relevant documentation on how to do that.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-22 18:20:05 +02:00
Martin Huschenbett
953b3c08fe
Save the expression in the CollectAuthority benchmark (#6454)
During some refactoring we forgot to save the initial expression to
evaluate for the machine during benchmarking. This PR fixes the issue.
It also make the error messages a bit more descriptive so that we can
actually debug this.

A test to make sure issues like this one don't get through CI again is
worked on by @gary-verhaegen-da in a separate PR.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-22 16:03:40 +00:00
Moritz Kiefer
cfb4c182a5
Consistently display stakeholders for key visibility errors (#6434)
* Consistently display stakeholders for key visibility errors

fixes #6404

As pointed out by Bernhard in #6404, the previous behavior was pretty
weird. If the committer was only a divulgee, we only displayed
stakeholders. If the committer was neither a stakeholder nor a
divulgee, we displayed stakeholders + parties the contract has been
divulged to. Given that only stakeholders can do lookups it makes much
more sense to display them consistently which is what this PR
achieves. I’ve also renamed “disclosed to” to “stakeholders” to make
it very explicit what is shown there.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>

* fmt

changelog_begin
changelog_end

* lalala

changelog_begin
changelog_end

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-06-22 10:22:37 +00:00
Moritz Kiefer
038388cdb7
Fix handling of non-visibile contract keys in scenario runner (#6433)
fixes #6403

I am not entirely sure why I thought that using `missingWith` makes
sense here but it clearly doesn’t make sense and resulted in a pretty
bad bug where a transaction both succeeded via `submit` as well as
failed via `submitMustFail` which is clearly the wrong thing to do.

This PR fixes this issue and introduces a `notVisibleWith` function
that does the right thing. I’ve also added some comments and an extra
assertion to clarify things a bit.

changelog_begin
changelog_end
2020-06-22 10:20:24 +02:00
Remy
1b1b4eab2c
Speedy: clean machine builder name (#6427)
* Address comment martin made in #6368

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-19 14:58:13 +02:00
Remy
5a9e7ebc7c
Speedy: refactor Machine builders (#6368)
In this PR we cleanup the constructor for the speedy Machine.

* We remove the `case`  keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
  + one generic builder `Machine.apply`,
  + scenario specific builder,

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-18 15:39:55 +02:00
nickchapman-da
61aa7039ef
Disable stack-traces in the speedy performance benchmarks. (#6358)
changelog_begin
changelog_end
2020-06-16 07:01:49 +00:00
Remy
f487b051f9
Ledger: use standard transactions in the indexer (#6297)
CHANGELOG_BEGIN
CHANGELOG_END
2020-06-11 16:12:21 +02:00
Remy
00972f38b5
DAML-LF: split com.daml.lf.types.Ledger (#6264)
We split the object  com.daml.lf.types.ledger in three:
- one part in `com.daml.lf.ledger.` (in //daml-lf/transaction) for the part relative to EventId (shares between scenario service and sandbox)
- one part in `com.daml.lf.ledger.` (in //daml-lf/interpreter) for the part common to Blinding and Scenario
- one part in `com.daml.lf.scenario.` (in //daml-lf/interpreter) for the part specific to Scenario

fixes #6260

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-08 17:29:10 +02:00
Moritz Kiefer
1c8691250d
Handle contract key failures due to visiblity in scenarios (#6165)
Previously, we just crashed the scenario service instead of throwing a
proper scenario error. This meant that you had to look at the
debugging output to figure out what is going wrong. This is both a
shitty UX and also inconsistent with how we handle this for fetch and
exercise on contract ids that are not visible. This PR adds a new
error type that matches the one for invisible contract ids.

changelog_begin

- [DAML Studio] Fetches and exercises of contract keys associated with
  contracts not visible to the submitter are now handled properly
  instead of showing a low-level error.

changelog_end

fixes #5903
2020-05-29 18:35:53 +02:00
Remy
2a05bb877e
Scenario service: fix visibility of contract keys. (#6123)
fixes #5924

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-27 16:27:50 +02:00
nickchapman-da
974ff45a08
Improve CollectAuthority perf benchmark (#6080)
The aim of the `CollectAuthority` benchmark is to track performance improvements made to the
DAML compiler & the speedy interpreter. Unfortunately the benchmark was spending at least
20% of the time being benchmarked outside of the speedy machine execution code, and
instead interacting with the ledger. We would like to minimize this as much as possible.

The solution is to cache the responses from the ledger made during the setup() run, and
replay them during the benchmark `run()`s.  To perform the caching, we no longer make use
of the scenario-interpreter, but instead have our own simplified runner, specialized for
this benchmark, and managing the cache.

using the cache speeds up the benchmark by a factor of about x1.25 (Which is the origin of
the claim above about 20% outside of speedy execution).

    Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
      95.188 ±(99.9%) 0.410 ms/op [Average]
      (min, avg, max) = (94.177, 95.188, 95.890), stdev = 0.472
      CI (99.9%): [94.779, 95.598] (assumes normal distribution)

    Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
      75.881 ±(99.9%) 0.288 ms/op [Average]
      (min, avg, max) = (75.077, 75.881, 76.491), stdev = 0.332
      CI (99.9%): [75.593, 76.169] (assumes normal distribution)

Because of how we run perf benchmark on CI -- by running the benchmark for now vs. some
fixed time in the past -- it ought to be fine to change the benchmark like this, although
we might need some help from Gary!

As well as improving the focus of this benchmark going forwards, we should also gain a
retrospective improvement on the speedup work already committed, since they will no longer
we dragged down by time which is outside of our control.

changelog_begin
changelog_end
2020-05-27 11:15:59 +00:00
Remy
9e456a1016
DAML-LF: Kill RelativeContractId (#5991)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-25 22:30:45 +02:00
nickchapman-da
44d0128f29
Add control to speedy compiler to choose if stack-trace support occurs at runtime. (#6070)
Choices for `stacktracing` are `NoStackTrace` / `FullStackTrace`.

Adapt code so the selection is made by the original caller:
- `engine`
- `scenario-service`
- `repl-service`
- `daml-script` runner
etc

Currently, all callers pass `FullStackTrace` (the existing behaviour), except for the
exploration dev-code: `daml-lf/interpreter/perf/src/main/scala/com/daml/lf/explore`.

The idea is that once this control is in place, we can discuss if we can change how we
might expose it to the user, and/or perhaps change the default behaviour to have
`stacktracing` disabled.

changelog_begin
changelog_end
2020-05-22 16:36:03 +01:00
nickchapman-da
9e5ae15e6d
Bazel rule to inspect dar file when compiling daml (#6067)
This change extends the bazel rule for `compile_daml` to add a stage which will pretty-print the generated .dar file, suitable for human inspection. The generated file is named with a `.dar.pp` suffix, and will only be generated if explicitly requested as a build target or listed as a dependency.

Make use of the new rule by demanding CollectAuthority.dar.pp when the perf benchmark is built.

changelog_begin
changelog_end
2020-05-21 11:06:51 +00:00
Remy
deedec9b9f
Engine: Remove optionality of contract ID Seeding. (#5966)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-13 20:29:14 +02:00
nickchapman-da
53b24793f4
Simplify and clarify the public interface to Speedy. (#5881)
* Simplify and clarify the public interface to Speedy.

- Remove `isFinal`. A client just uses `run()`.
- Remove `toSValue`. The value in available in `SResultFinalValue(v: SValue)`.
- A client never directly access the `.ctrl` (or `.returnValue`) components.
- A client may use `setExpressionToEvaluate(expr)` to evaluate a new expression on an existing machine.

changelog_begin
changelog_end

* remove while loop which executes just once

* avoid unnecessary mutation when running speedy
2020-05-07 08:55:16 +00:00
nickchapman-da
b318cb0299
Optimize Speedy machine.ctrl (#5811)
Remove the `Ctrl` trait and separate `Machine.ctrl: Ctrl` into `Machine.ctrl: SExpr` and `Machine.returnValue: SValue` instead. This allows for avoiding dynamic dispatch on `ctrl` and instead allows for checking a pointer for `null` to decide if we have an expression that needs further break-down or a return value ready to be passed to the next continuation.

To make this check really only a pointer comparison we also needed to remove the abomination of "fully applied partially applied primitives". In order to achieve this, we check whether a PAP will be fully applied afterward when applying the last argument.

On the `collect-authority` benchmark, this increases throughput by around 13%, on another more computation heave benchmark by about 21%.

`collect-authority` benchmark on `master`:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
  112.361 ±(99.9%) 1.965 ms/op [Average]
  (min, avg, max) = (107.047, 112.361, 120.745), stdev = 3.493
  CI (99.9%): [110.396, 114.326] (assumes normal distribution)
```

`collect-authority` benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
  98.196 ±(99.9%) 1.933 ms/op [Average]
  (min, avg, max) = (91.580, 98.196, 105.478), stdev = 3.436
  CI (99.9%): [96.263, 100.129] (assumes normal distribution)
```

computation heavy benchmark on master
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
  44.030 ±(99.9%) 0.742 ms/op [Average]
  (min, avg, max) = (42.124, 44.030, 46.781), stdev = 1.319
  CI (99.9%): [43.289, 44.772] (assumes normal distribution)
```

computation heavy benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
  36.222 ±(99.9%) 0.580 ms/op [Average]
  (min, avg, max) = (34.897, 36.222, 39.787), stdev = 1.031
  CI (99.9%): [35.643, 36.802] (assumes normal distribution)
```

changelog_begin
changelog_end
2020-05-06 20:44:50 +02:00
nickchapman-da
572b21e882
Speedy: run() dont step() (#5814)
Speedy: run() dont step()

- Running the Speedy machine with  `run()` instead of `step()`
- Remove: `SResultContinue`
- Add: `SResultFinalValue(_)`

We change the top level control of Speedy: from machine.step() to machine.run, with the control of stepping while the machine returns SResultContinue moved into speedy itself. (And so SResultContinue is removed in favour of SResultFinalValue.) The main advantage of this approach is that the tight while loop can be moved inside the exception handler, rather than having to wrap the handler every step.

changelog_begin
changelog_end
2020-05-04 16:57:47 +01:00
Martin Huschenbett
cebc26af88
Make DAML execution benchmarks more flexible (#5771)
Instead of always benchmarking the hardcoded scenarios, we can now
pass a DAR and a scenario to the benchmark as well.

This is part of https://github.com/digital-asset/daml/issues/5746.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-29 12:41:43 +02:00
Martin Huschenbett
ea50c329f2
DAML Exec Perf: Add simple benchmark for scenario interpreter (#5747)
We add a benchmark for running a sufficiently big scenario. This PR is
more about setting up the benchmarking infrastructure rather than the
actual benchmark itself.

This is part of https://github.com/digital-asset/daml/issues/5746.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-28 14:44:29 +02:00
Remy
0f75bf3fde
Scenario service: fails when trying to lookup not visible key (#5561)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-23 21:55:39 +02:00
Remy
15354c3256
DAML-LF: remove submitter is in maintainer check (#5611)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-23 16:10:39 +02:00
Remy
9704a39bae
Engine: refactor PartialTransaction context (#5578)
* Engine: refactor PartialTransaction context

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-16 14:39:12 +02:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Stephen Compall
8d2c13cf43
generating GenMaps for testing (#5115)
* building a GenMap generator

* move Party Order to accessible place; test whether Name Equal is still in use

* add Order instance for SortedLookupList

* switch to Map for genMap's Inj; more Order instances

* remove all Order from TypedValueGenerators

* Revert "add Order instance for SortedLookupList"

This reverts commit 03a59a8249.

* moving the Equal instance means scenario-interpreter no longer direct-deps scalaz

* add a test using TypedValueGenerators GenMap

* remove Party Order

* refmt bazel

* remove stray import

* followup dep change to moving the Equal[Name] instance

* add changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Name equal instance appears to be no longer used, but keeping anyway
2020-03-25 10:28:06 -04:00
Remy
eae3844ed9
Engine: Replace SResultMissingDefinition by SResultNeedPackage (#5039)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-17 19:25:57 +01:00
Remy
93a613ff42
daml-lf: cleanup StringIds (#4221)
clean up String Identifier in daml-lf 

- Separate LedgerString from ContractIdString
- Drop TransactionIdString from daml-lf 
- Create a new ContractIdString (aka. ContractIdStringV1) for new contractId comparable with  relative contractIds

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-04 09:30:53 +01:00
Andreas Herrmann
f33e79c787
Remove unused dependencies to da_scala_library (#3938)
* Inline all scala_library dependencies

* Run //:buildifier-fix

* TMP scala_library_suite --> scala_library

* da_scala_library: Enable unused dependency checker

* scala_library: Enable unused dependency checker

* //daml-lf/data:data

* //daml-lf/engine:engine

* //ledger-api/rs-grpc-akka:rs-grpc-akka

* //ledger/participant-state:participant-state

* //ledger/ledger-api-client:ledger-api-client

* //scala-protoc-plugins/scala-logging:scala-logging-lib

* //ledger/ledger-api-scala-logging:ledger-api-scala-logging

* //ledger/ledger-api-common:ledger-api-common

* //ledger-service/utils:utils

* //ledger-service/jwt:jwt

* //ledger/ledger-api-auth:ledger-api-auth

* //extractor:extractor

* //daml-assistant/scala-daml-project-config:scala-daml-project-config

* //language-support/codegen-common:codegen-common

* //language-support/scala/codegen:codegen

* //language-support/codegen-main:codegen-main-lib

* //ledger-service/db-backend:db-backend

* //ledger-service/http-json:http-json

* //daml-lf/scenario-interpreter:scenario-interpreter

* //ledger/sandbox:sandbox

* //navigator/backend:navigator-library

* //daml-assistant/daml-sdk:sdk-lib

* //daml-lf/data-scalacheck:data-scalacheck

* //daml-script/test:test-lib

* //ledger/ledger-api-common:ledger-api-common-scala-tests-lib

* //ledger/test-common:test-common

* //ledger/sandbox:sandbox-scala-tests-lib

* //extractor:extractor-scala-tests-lib

* //language-support/java/bindings:bindings-java-tests-lib

* //language-support/java/bindings-rxjava:bindings-java-tests-lib

* //language-support/scala/bindings-akka-testing:bindings-akka-testing

* //language-support/scala/codegen-testing:codegen-testing

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app

* //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app-testing

* //language-support/scala/codegen-testing:codegen-testing-testing

* //ledger-api/sample-service:sample-service

* //ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-lib

* //ledger/ledger-api-test-tool:ledger-api-test-tool-tests

* //ledger/participant-state/kvutils:kvutils

* //ledger/sandbox:ledger-api-server

* //ledger/sandbox-perf:sandbox-perf-lib

* //navigator/backend:navigator-tests-library

* UNDO scala_library_suite --> scala_library

This reverts commit ab3eb1ae23139e2ec431ab4551fbb0371e0354e1.

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 18:14:21 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Andreas Herrmann
9fbb787062 Remove unused dependencies to da_scala_test(_suite) (#3925)
* Remove unused scala.bzl imports

* override_targets org.scalatest.scalatest_2.12

Otherwise, rules_scala implicitly adds a different version to scala_test
than other packages transitively depending on scalatest. This causes
unused dependency checker to raise an error.

* Handle rules_scala scalatest in pom_file.bzl

* Inline all scala_test dependencies

So that `unused_dependency_checker = "error"` can be applied to them.

* Run //:buildifier-fix

* TMP scala_test_suite --> scala_test

* da_scala_test: Enable unused dependency checker

* //navigator/backend:navigator-scala-tests

* //ledger/sandbox:sandbox-scala-tests

* //ledger/participant-state/kvutils:kvutils-tests

* //ledger/participant-state:participant-state-tests

* //ledger/ledger-api-scala-logging:ledger-api-scala-logging-test

* //ledger/ledger-api-common:ledger-api-common-scala-tests

* //ledger/ledger-api-client:ledger-api-client-tests

* //ledger/ledger-api-auth:ledger-api-auth-scala-tests

* //ledger-service/lf-value-json:tests

* //ledger-service/jwt:tests

* //ledger-service/http-json:tests

* //ledger-api/rs-grpc-akka:rs-grpc-akka-tests

* //language-support/scala/codegen-sample-app:tests

* //language-support/scala/codegen-sample-app:ScalaCodeGenIT

* //language-support/scala/codegen:tests

* //language-support/scala/bindings-akka:tests

* //language-support/java/codegen:test

* //language-support/java/codegen:ledger-tests

* //language-support/java/bindings-rxjava:bindings-java-tests

* //language-support/codegen-common:test

* //extractor:extractor-scala-tests

* //daml-lf/scenario-interpreter:scenario-interpreter_tests

* //daml-lf/language:language-test

* //daml-lf/interface:tests

* //daml-lf/engine:tests

* //daml-lf/encoder:tests

* //daml-lf/archive:daml_lf_archive_reader_tests

* //daml-assistant/scala-daml-project-config:scala-daml-project-config-tests

* UNDO scala_test_suite --> scala_test

This reverts commit 13ed47ba725e944533ca1157a070cb8dc30569ac.

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2019-12-30 13:49:59 +00:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
Francesco Mazzoli
ca5d045e62 check that submitter is in maintainers for lookup by key (#1967)
* check that submitter is in maintainers when looking up keys

Fixes #1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.

* add UNTIL-LF to run tests up to a certain version of DAML-LF

* name targets for DAML tests better

* add notes about DAML-LF changes

* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly

* add scenario tests for #1866

* add warnings about future key behavior in docs

* use flag rather than version when executing
2019-07-05 09:34:26 +00:00
Ognjen Maric
54cf2ee836
Retain fetch actors during reinterpretation (#1414)
DAMLe: add submitters and retain fetch node actors in reinterpretation

Co-Authored-By: Francesco Mazzoli <f@mazzo.li>
2019-06-28 15:40:52 +02:00
Remy
0591075187 cleanup daml-lf scala packages (#1581)
* cleanup daml-lf scala packages

* Address Stephen's Comments

* update maven coordinates of language package
2019-06-12 15:55:48 +00:00
Remy
4f18b1afa7 DAML-LF internal type safety (#1192)
* Rename Value's ContractId to VContractId

* daml-lf: a bit more about PartyId

* daml-lf: Concatenable MatchingStringModule

* daml-lf make clear type used for Scenarios only

* daml-lf create ContractId, LedgerId, TransactionId

* sandbox-sql conversion util

* LedgerName -> LedgerString

* futher type cleanup in the sandbox

* daml-lf add test for LedgerString

* fixing tests

* a bit more safety in the DB

* Address Stephen's comments

* fix rebase

* More fixes for StringModule

* change length of LedgerString (256 -> 255)
2019-05-24 09:53:29 +00:00
Remy
c7df212d42 Daml lf type safty (Decimal) (#1098)
* daml-lf: make Decimal type safe

* daml-lf: create Utf8String type

* daml-lf: cleanup in data package

* Address Stephen Comments

* daml-lf: remove UTF8String
2019-05-14 18:10:45 +00:00
Remy
2e3a87934b Daml lf type safty (ChoiceName, VarName, FieldName, ConstructorName) (#983)
* daml-lf: make DefinitionRef more typesafe

* daml-lf: Identifier -> DefinitionRef

* daml-lf: remove unsafe apply and copy methods from DottedName

* daml-lf: create identifier

* daml-lf: make ChoiceNames Identifiers

* daml-lf: cleanup TVar

* daml-lf: FieldNames & VariantConstructors -> Identifiers

* bazel fmt

* daml-lf: VarName -> Identifier

* daml-lf: drop return inside Ref.scala

* daml-lf Identifier -> Name

* daml-lf DefinitionRef -> Identifier

* daml-lf make iface more type safe
+ address Francesco's comments

* daml-lf: remove unsafe unapply from MatchingStringModule

* fix navigator

* Address Stephen's Comments
2019-05-13 11:17:12 +00:00
Remy
0489c6e0a5
Daml lf type safty (Party & PackageId) (#761)
* daml-lf: split SimpleString into Party and PackageId

* daml-lf remove parameter from DefinitionRef
2019-05-06 20:40:43 +02:00
gleber
b3e2e10897 scenario-tester: Allow to mangle names used in scenarios before their executions. (#795)
* Allow to mangle names used in scenarios before their executions.

This allows to run a scenario against a long-running server repeatedly and avoid
clashes between runs, since each party is unique (up to the randomness used).

* ScenarioRunner: add test for partyNameMangler
2019-05-03 13:04:23 +00:00
Francesco Mazzoli
cf89366ac6 respect contract visibility when looking up contract keys (#753)
* remove outdated flags from comments

* update tests in preparation for visibility rules

* check visibility of contract keys, fixes #751

* add release notes

* include visibility check in scenario runner

* stephen's suggestions

* fix type error in api server example

* scalafmt

* bring TransactionFiltration.scala up to speed with disclosure

see
<faf1cde1a1/ledger-api/grpc-definitions/com/digitalasset/ledger/api/v1/event.proto>
for an in-depth description. fixes #794.

* add tests for contract key visibility

* move a lot of ledger testing utils out of command checks

* move a lot of ledger testing utils out of command checks

* remove useless checks regarding divulgence

* add tests regarding event witnesses

* update release notes

* re-introduce safety when disclosing
2019-05-02 14:02:30 +00:00
gleber
aa70c7f64e
Enforce consistent formatting of BUILD files. (#412)
* Add buildifier targets.

The tool allows to check and format BUILD files in the repo.

To check if files are well formatted, run:

    bazel run //:buildifier

To fix badly-formatted files run:

    bazel run //:buildifier-fix

* Cleanup dade-copyright-headers formatting.

* Fix dade-copyright-headers on files with just the copyright.

* Run buildifier automatically on CI via 'fmt.sh'.

* Reformat all BUILD files with buildifier.

Excludes autogenerated Bazel files.
2019-04-12 13:10:16 +02:00
Jussi Mäki
1f2246c822 Do not divulge contracts to observers in nonconsuming exercises (#325)
* Do not divulge contracts to observers in nonconsuming exercises

Disables support for non-default ledger feature flags, as they
are meaningless since ledger server logic does not respect the flags.
Instead of large refactoring to add support for the old flag settings,
it is best to disallow the deprecated flags, and later on phase out the
flags completely.

Re-enables test_divulgence_of_token in sandbox semantic tests.

Fixes #157.

* purge LedgerFlags entirely...

...since we only support one version of them anyway, and clearing them

* updated release notes
2019-04-11 14:41:35 +02:00
moritzkiefer-da
fa4067ad1b
Move POM file generation to Bazel rules (#374)
* Move POM file generation to Bazel rules
2019-04-11 11:24:52 +02:00
Digital Asset GmbH
05e691f558 open-sourcing daml 2019-04-04 09:33:38 +01:00