Commit Graph

51 Commits

Author SHA1 Message Date
Moritz Kiefer
2c08586d33
Expose Daml stacktraces for Daml Script errors (#9025)
* Expose Daml stacktraces for Daml Script errors

This finally plugs together the pieces from the previous PRs to
provide stacktraces on any ScriptF command (and thereby anything
involving an interaction with the ledger).

fixes #8754

changelog_begin

- [Daml Script] When running Daml Script on the command line you will
  now see a Daml stacktrace on failures to interact with the ledger
  which makes it significantly easier to track down which of the calls
  fails. By default, you will only get the callsite of functions like
  `submit`. To extend the stack trace, add `HasCallStack` constraints
  to functions and those will also be included.

changelog_end

* Fix non-determinism in tests

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-03-04 16:13:50 +01:00
Moritz Kiefer
ec36066986
Split LedgerInteraction into different files (#8975)
Pure shuffling around there is no logic change in here. I keep getting
lost in the 1.4k LedgerInteraction file so this client splits it up
into a 4 different files to make it a bit easier to navigate.

changelog_begin
changelog_end
2021-03-01 16:27:34 +01:00
Moritz Kiefer
275eaedd2a
Fix closure references in Daml Repl (#8950)
* Fix closure references in Daml Repl

Turns out the comment "we probably need to extend this to merge the
modules from each line" was exactly correct: If the result evaluates
to a closure (or a value including a closure), it can reference
definitions from the current module. This happens if the simplifier
lifted something out of the current definition (otherwise we have only
one and it cannot be recursive so no chance of leaking a reference).

This PR fixes this by checking whether the result references the
current module and if it does, we keep it around.

changelog_begin
changelog_end

* Address review comments

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-02-26 13:29:42 +00:00
Moritz Kiefer
773ddada04
Upgrade nixpkgs (#8190)
* Upgrade nixpkgs

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* No ibazel

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* Switch agent pool

changelog_begin
changelog_end

* .

* .

changelog_begin
changelog_end
2021-02-08 11:12:07 +00:00
Moritz Kiefer
5d91a25306
Disable deriving-defaults per file (#8684)
Disabling it per target works nicely for compilation but it gets
annoying in ghci since the warnings are still triggered. We could
disable it everywhere but I think the warning is generally useful. I
tried patching proto3-suite to use DerivingStrategies but that doesn’t
work because haskell-src is dead and doesn’t support it. So for now
adding it to the per-file list seems like the best option.

changelog_begin
changelog_end
2021-01-29 11:01:55 +00:00
Moritz Kiefer
e0c5abd647
Switch to GHC 8.10.3 (#8394)
* Switch to GHC 8.10.3

changelog_begin
changelog_end

* Update bazel-haskell-deps.bzl

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Comment on rules_haskell patch

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2021-01-25 11:53:53 +00:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Moritz Kiefer
424faa923a
Port damlc dependencies to Scala 2.13 (#8423)
* Port damlc dependencies to Scala 2.13

I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).

So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).

If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.

changelog_begin
changelog_end

* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* fixup lawlesstraversal

changelog_begin
changelog_end

* less iterator more view

changelog_begin
changelog_end

* document safety of unsafeWrapArray

changelog_begin
changelog_end

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-01-08 07:22:38 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Remy
009b030463
LF: Simplify Language Version representation (#8258)
We drop the distinction (at the type level) of Dev and Stable language
version.  The two main reason that motivate this choice:
* we never really used this distinction.
* we want to add the concept of "preview" version (which is neither Dev nor Stable)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:21:45 +01:00
Jussi Mäki
fa9bc4a984
Cache computation of top-level values at definition level (#7818)
* Cache computation of top-level values at definition level

Earlier the computation of a top-level value was only cached at the
use-site in SEVal. This introduces SDefinition which contains the same
mechanism as SEVal to cache the computation.

As expected this does not impact performance much:

before: CollectAuthority.bench  //daml-lf/scenario-interpreter/CollectAuthority.dar  CollectAuthority:test  avgt   40  44.267 ± 0.728  ms/op
after: CollectAuthority.bench  //daml-lf/scenario-interpreter/CollectAuthority.dar  CollectAuthority:test  avgt   40  43.693 ± 0.702  ms/op

What this does have a significant impact is on reducing the number of distinct
SValues for things like type class dictionaries etc, so that now we have one
SValue per dictionary rather than one per SEVal.

CHANGELOG_BEGIN
CHANGELOG_END

* Address code review

* Fix speedy tests
2020-10-28 13:39:20 +00:00
Remy
dc34d4f637
LF: add PackageInterface a lightweigh Package (#7577)
CHANGELOG_BEGIN
CHANGELOG_END
2020-10-13 12:41:22 +02:00
Remy
76ef4c861a
Speedy: group compiler parameters into a Config case class. (#7438)
* Speedy: group compiler parameters into a Config case class.

Speedy compiler is parametrized by several option flags.  Those flags
are passed through different calls (in particular in the instance of
`CompiledPackages`.

This PR group all this configuration parameters into a case class to
pass the option in a cleaner way.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-18 19:32:26 +02:00
Moritz Kiefer
4460cb5e08
Fix trace statement handling in DAML Script service (#7292)
* Fix trace statement handling in DAML Script service

This PR makes sure that we get both trace statements coming from the
ledger client as well as from the ledger server. In the script service
those are two different Speedy machines so we need to interleave
them. This is a bit messy so let me explain the steps (apologies for
not splitting it up but I think it’s easier to understand the
motivation when it is a single PR):

1. Abstract over the tracelog interface. The ringbuffer doesn’t makes
   sense for our purposes and is annoying to clear.

2. Pass in the respective fields of `Machine` to `Conversions` instead
   of the whole machine. This is both a simpler design (passing around
   mutable objects scares me) and it allows us to assemble the fields
   from different machines.

3. Initialize the ledger machine with a simple ArrayBuffer tracelog.

4. After `submit` and `submitMustFail` copy the tracelog from the
   ledger to the client.

fixes #7280

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2020-09-01 17:15:48 +00:00
Andreas Herrmann
037cf11710
Add :json command to DAML REPL (#7202)
* REPL test `:json` command

changelog_begin
changelog_end

* Implement :json command in DAML REPL

changelog_begin
- [DAML REPL] You can now convert DAML expressions to JSON in the DAML
  REPL using the meta-command ``:json`` for example ``:json [1, 2, 3]``.
changelog_end

* Extend DAML REPL documentation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-21 14:53:06 +00:00
Moritz Kiefer
0e833c02a4
Enable NonUnitStatements for repl service (#7161)
changelog_begin
changelog_end
2020-08-18 10:58:00 +02:00
Moritz Kiefer
e1236d42ed
Make ApplicationId in DAML Script configurable (#7074)
This is set per participant since it is similar to the token file.

fixes #7029

changelog_begin

- [DAML Script/DAML REPL] You can now configure the application id via
  `--application-id` or the `--participant-config`. This is primarily
  useful if you are working against a ledger with authentication and
  need to match the application id in your token.

changelog_end
2020-08-10 17:24:38 +02:00
Moritz Kiefer
733a43aeea
Move ApplicationId in daml-script to ledger client (#7070)
This makes much more sense since for things like the JSON API and the
script service, we don’t actually have the application id as a
separate parameter.

I’ve also cleaned up all the arbitrary hardcoded application id in
various tests in favor of a DEFAULT_APPLICATION_ID value.

Next step is #7029

changelog_begin
changelog_end
2020-08-10 13:22:51 +02:00
Remy
17926c5a8a
LF: move language version from modules to packages (#7064)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-10 11:26:58 +02:00
Stephen Compall
96624a7677
use -Ywarn-unused for all Scala code (#6907)
* add -Ywarn-unused to all scalac options

* remove some unused arguments

* remove some unused definitions

* remove some unused variable names

* suppress some unused variable names

* changeExtension doesn't use baseName

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* work around no plugins in scenario interpreter perf tests

* remove many more unused things

* remove more unused things, restore some used things

* remove more unused things, restore a couple signature mistakes

* missed import

* unused argument

* remove more unused loggingContexts

* some unused code in triggers

* some unused code in sandbox and kvutils

* some unused code in repl-service and daml-script

* some unused code in bindings-rxjava tests

* some unused code in triggers runner

* more comments on silent usages

- suggested by @cocreature; thanks

* fix missing reference in TestCommands

* more unused in triggers

* more unused in sandbox

* more unused in daml-script

* more unused in ledger-client tests

* more unused in triggers

* more unused in kvutils

* more unused in daml-script

* more unused in sandbox

* remove unused in ledger-api-test-tool

* suppress final special case for codegen unused warnings

.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
      implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                         ^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
      implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                               ^

* one more unused in daml-script

* special scaladoc rules may need silencer, too

* unused in compatibility/sandbox-migration

* more commas, a different way to `find`

- suggested by @remyhaemmerle-da; thanks
2020-08-07 13:16:09 -04:00
Moritz Kiefer
5ceeb476c8
Remove duplicated tls config parsing from DAML Script/Triggers (#6947)
This was not only unnecessarily duplicated, it also had a bug where
`--crt` behaved like `--pem` instead of setting the cert chain.

I didn’t add new tests since it seems like the wrong place to test
config parsing of a library. We do have tests for TLS in general for
both DAML Script and DAML Triggers.

changelog_begin
changelog_end
2020-07-31 18:25:18 +02:00
Moritz Kiefer
9881ff5231
Make DAML REPL work without a ledger (#6838)
This PR makes the ``--ledger-host`` and ``--ledger-port`` parameters
optional so DAML REPL works without a ledger which is useful now that
we have better. support for pure expressions. This just piggybacks on
DAML Script’s multiparticipant support so there are no significant
changes on the service.

Docs are updated and we have a testcase.

Side note: What is still missing is `let x = …` in DAML REPL. I’ll
tackle that in a separate PR.

changelog_begin
changelog_end
2020-07-23 10:23:24 +00:00
Andreas Herrmann
4b1438276c
Update Bazel 2.1.0 --> 3.3.1 (#6761)
* Upgrade nixpkgs revision

* Remove unused minio

It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.

* Update Bazel on Windows

changelog_begin
changelog_end

* Fix hlint warnings

The nixpkgs update implied an hlint update which enabled new warnings.

* Fix "Error applying patch"

Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.

* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681

Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.

```
git diff ':(exclude)WORKSPACE'
```

* Update rules_nixpkgs

* nixpkgs location expansion escaping

* Drop --noincompatible_windows_native_test_wrapper

* client_server_test using sh_inline_test

client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.

This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.

* daml_test using sh_inline_test

* daml_doc_test using sh_inline_test

* _daml_validate_test using sh_inline_test

* damlc_compile_test using sh_inline_test

* client_server_test find .exe on Windows

* Bump Windows cache for Bazel update

Remove `clean --expunge` after merge.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-23 09:46:04 +02:00
Moritz Kiefer
6bf1eb19ae
Print results in DAML REPL and support pure expressions (#6799)
* Print results in DAML REPL and support pure expressions

This extends DAML REPL to behave similar to GHCi in that it prints
results if the result is an instance of `Show` and it also supports
pure, non-script expressions now. We copy the same hack from GHCi and
typecheck multiple times. This doesn’t seem to have any noticeable
performance impact (which makes sense, the expressions are tiny and
the overhead of grpc is much much bigger).

Docs and tests are updated.

fixes #6780

changelog_begin
changelog_end

* windows is garbage

* but I fixed it \o/

* Rename getReplLogger to newReplLogger

changelog_begin
changelog_end
2020-07-21 09:01:18 +00:00
Gary Verhaegen
8176fb0c8d
fix the the typo (#6723)
```
s/the the /the /
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 16:55:33 +00:00
Moritz Kiefer
02c59d4f2a
Revert ANF changes and add a testcase for evaluation order (#6645)
* Revert ANF changes and add a testcase for evaluation order

After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.

Therefore, this PR reverts the following commits:

1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee

Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably

1. https://github.com/digital-asset/daml/pull/6576 which was easy to
   resolve and the change to return SEValue instead of SExpr is still
   nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
   some changes since the constructors changed. If you want to review
   those changes in detail (they are pretty straightforward so not too
   important), it’s probably easiest to check out this PR and run
   ```
   git diff 2cd2a8f2a8
   daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
   ```
   to see the diff to the parent commit of the first commit that
   introduced ANF.

changelog_begin
changelog_end
2020-07-08 12:40:12 +02:00
nickchapman-da
353d0da6f7
Speedy ANF (#6440)
* ANF transformation in Speedy.

The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.

changelog_begin
changelog_end

* improve comment

* inline functions relocateA/L

* remove comment about scalafmt

* remove commented out alterative def for transformLet1

* improve code by adding incr methods to DepthA/E

* remove (n == 0) special case in trackBindings

* clarify comment further

* improve validate/go to not consume stack for deeply right-nested let-expressions

* address comments from Remy: be private; use final case case; etc

* rename to unsafeCompilationPipeline

* add back some trailing commas

* remove commented-out debug line

* improve comment

* remove dev/debug code in compilationPipeline

* remove commented out code in SEAppGeneral.execute

* undo unrelated code improvement in SValue.scala

* fix compile. object Anf cannot be private
2020-06-30 20:52:44 +00:00
Andreas Herrmann
a51c4d1d2e
DAML REPL support static-time mode (#6485)
* DAML REPL --static-time regression test

* damlc repl --static-time|--wall-clock-time flags

CHANGELOG_BEGIN
- [DAML REPL] The time mode can now be specified using the
  ``--static-time`` and ``--wall-clock-time`` flags.
CHANGELOG_END

* Update compiler/damlc/lib/DA/Cli/Damlc.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Verify the effect of setTime using getTime

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-06-25 13:08:10 +00:00
Moritz Kiefer
97c74d4a29
Support multiple auth tokens in DAML Script (#6473)
* Support multiple auth tokens in DAML Script

This piggy backs on top of the already existing --participant-config
feature. While you can argue that it might be slightly confusing that
you have to specify the same participant twice to specify different
auth tokens, I think this actually makes sense: In an ideal
world (ignoring any performance issues) you have one participant per
party anyway and one connection per participant specified in the
config file still seems like a very reasonable model.

changelog_begin

- [DAML Script] You can now use DAML Script with multiple auth
  tokens. This is particularly useful if you are working with the JSON
  API where you can only have one party per token or with an IAM that
  only provides single-party tokens. The tokens are specified in the
  participant configuration passed via `--participant-config` in a new
  ``access_token`` field. The existing `--acess-token-file` flag is still supported if you want to use the same token for all connections. See
  https://docs.daml.com/daml-script/index.html#running-daml-script-against-authenticated-ledgers
  for more details.

changelog_end

* I will never understand rst

changelog_begin
changelog_end
2020-06-24 14:43:29 +02:00
Moritz Kiefer
0a8c1eba4e
Integrate time service into DAML Script (#6417)
This integrates the time service into DAML script thereby covering the
main piece of scenarios that was missing from DAML script.

This PR does two things (they are very related and doing them together
makes it much easier to test):

1. It “fixes” `getTime` to return the ledger time in static mode by
   querying the ledger time service instead of defaulting to the Unix
   epoch which is pretty useless and I would consider the old behavior
   a bug. We keep the old behavior via the JSON API since there is no
   time service.

2. It adds `setTime` to set the ledger time via the time service. This
   is only supported in static time mode (sadbonx and other ledgers do
   not expose the time service in wallclock mode because changing time
   makes it not wallclock) or via the JSON API (no time service).

fixes #6220

changelog_begin

- [DAML Script] DAML Script’s `getTime` now correctly handles time
  changes in static time mode and returns the current time by querying
  the time service rather than defaulting to the Unix epoch. Note that
  when run via the JSON API, it still returns the Unix epoch.

- [DAML Script] Add `setTime` to DAML Script which sets the ledger
  time via the ledger API time service. Note that this is only
  supported when running over gRPC in static time mode.

changelog_end
2020-06-18 18:25:24 +02:00
Simon Maxen
5329a4f97f
Ledger id requirement (#6323)
* Add option based constructor for LedgerIdRequirement

changelog_begin
changelog_end

* Make option based consructor the default, deprecate old constructor

* Update with review comments
2020-06-12 15:00:32 +00:00
nickchapman-da
44d0128f29
Add control to speedy compiler to choose if stack-trace support occurs at runtime. (#6070)
Choices for `stacktracing` are `NoStackTrace` / `FullStackTrace`.

Adapt code so the selection is made by the original caller:
- `engine`
- `scenario-service`
- `repl-service`
- `daml-script` runner
etc

Currently, all callers pass `FullStackTrace` (the existing behaviour), except for the
exploration dev-code: `daml-lf/interpreter/perf/src/main/scala/com/daml/lf/explore`.

The idea is that once this control is in place, we can discuss if we can change how we
might expose it to the user, and/or perhaps change the default behaviour to have
`stacktracing` disabled.

changelog_begin
changelog_end
2020-05-22 16:36:03 +01:00
Martin Huschenbett
caedc72551
Implement a simple profiler for DAML scenarios (#5957)
* Implement a simple profiler for DAML scenarios

The profiler runs a single scenario and records timing information when
each function (and some other closures) are entered and left. The
resulting information can be visualized as a flamegraph using
[speedscope](https://www.speedscope.app/).

The profiler works by instrumenting the CEK machine at the heart of
DAML Engine. Unfortunetaly, this causes a very small overhead on
non-profiling runs too. However, in my benchmarks I could not measure
any significant impact on the overall runtime at all. More precisely,
the overhead is as follows:

Every closure now has an additional field called `label`. In
non-profiling runs this field is always set to `null`. This field needs
to be allocated, copied whenever we copy a closure and scanned during
garbage collection. Additionally, whenever we enter a closure, we check
this field and whenever it is _not_ `null`, i.e. never during
non-profiling runs, we record an "open event" and set up a hook for the
corresponding "close event". Thus, the additional cost during
non-profiling runs are a single pointer comparison and a jump beyond
the "then branch".

Since this is still very much in active development, there are no
documentation, other than an entry in a README, and no tests yet. They
will come before we promote this. However, the UX will look very
different then since we already have plans to significantly change it.

CHANGELOG_BEGIN
CHANGELOG_END

* Run scalafmt

* Make profiling argument to PureCompiledPackges optional

* Fix a bunch of tests

CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt is so annoying

CHANGELOG_BEGIN
CHANGELOG_END

* Apply simple suggestions

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-18 17:20:44 +00:00
Moritz Kiefer
2853a20eb9
Make max-inbound-message-size in DAML script/repl configurable (#5996)
Fixes #5592

The CLI syntax and the defaults follow the JSON API here.

changelog_begin

- [DAML Script] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON
API.

- [DAML REPL] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON API.

changelog_end
2020-05-15 07:22:05 -04:00
Remy
861724e4fd
Engine: cleanup SEVal (#5859)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-07 11:44:19 +02:00
Remy
7bdde70840
Speedy Compiler: clean up. (#5649)
CHANGELOG_BEGIN
CHANGELOG_END
2020-04-22 16:09:54 +02:00
Moritz Kiefer
d83387a64d
Improve error messages in daml repl on calls to error (#5565)
* Improve error messages in daml repl on calls to `error`

There were two issues with calls to `error`:

1. This one is harmless but somewhat annoying: When calling `error` we
   run into the log statement in `stepToValue` which prints out the
   error message in a fairly reasonable form (you can argue whether
   Error: User abort: is a super useful prefix but that’s a relatively
   minor issue). Afterwards we then call `println` on the failed
   future. However, that will just print the type of the exception
   which isn’t something we want to show to users. I’ve just disabled
   the println statement if the exception is `SError`.

2. This one is a bigger issue: `throw x` is not the same as
   `Future.failed(x)`. I only fully realized the difference fairly
   recently. The former fails before it produces a future. So `(throw
   x).onComplete(…)` will never execute the callback. The latter is
   just a failed future. It is rather confusing to have a function
   that returns a future but then throws an exception instead of a
   future and it confuses the grpc library which prints out a horrible
   exception. I’ve changed all calls to `throw` in `runWithClients` to
   instead use `Future.failed` and `flatMap` (in the form of
   for-comprehensions).
   There are still a few calls in `run` left which I’ll leave for a
   separate PR.

I think we need to factor out some helper functions here to make this
a bit more manageable (e.g. for the Converter.toFuture) stuff but I’ll
leave that for a separate PR. You probably want to view this with
whitespace diffs disabled.

changelog_begin

- [DAML Repl] DAML Repl now produces better error messages on calls to
  `error` and `abort`.

changelog_end

* Switch stepToValue to return Either
2020-04-15 16:19:36 +00:00
Moritz Kiefer
6af5b97273
Split up repl tests to make them faster (#5450)
* Split up repl tests to make them faster

This PR splits up the tests into the tests for TLS and Auth and the
tests for the actual functionality.

The func tests use the repl as a library which allows them to be
significantly faster:

1. We only need to start the service process once.
2. We only need to initialize the package db once.

There is at least one other point that I did not address for now and
that is only loading the packages into the repl service once. While
loading them multiple times is a noop, it still has a performance
implication.

Sadly, this has turned out much more messy than I thought it would be
due to various issues with haskeline/repline/tasty/computers. The
details are in a comment in DA.Test.Repl.FuncTests.

changelog_begin
changelog_end

* Try to nuke cache on window selectively

* Enable cache again
2020-04-07 10:22:15 +02:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Andreas Herrmann
36c3c95933
Regression test for silent REPL server errors (#5347)
* Regression test for silent REPL server errors

CHANGELOG_BEGIN
CHANGELOG_END

* Fix script error test case

We get an error print out by the REPL server as well

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-03 09:59:06 +00:00
Andreas Herrmann
90c94b0076
Use repline for DAML REPL (#5337)
* Use repline for DAML REPL

CHANGELOG_BEGIN
CHANGELOG_END

* REPL server print error messages

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-01 13:13:46 +00:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Moritz Kiefer
46be910ba1
Abstract over ledger interaction in DAML script (#5184)
This PR adds as `ScriptLedgerClient` trait (happy to change the name
if anyone has a better proposal) that abstracts over the interaction
with the ledger. This will allow us to plug in a different
implementation for interacting with the JSON API so we can run DAML
scripts against DABL or other environments where gRPC is not a
workable option. Note that this PR does not yet add the implementation
for interacting with the JSON API. I’ll leave that for a separate PR.

changelog_begin
changelog_end
2020-03-25 15:26:18 +01:00
Robert Autenrieth
3f597aae16
New ledger time (#5100)
* Tighten result type

Command execution can't result in a sequencer error

* New helper method for extracting used contracts

* New error clause

* Add a DAO query for the maximum time of contracts

* Implement algorithm for finding ledger time

CHANGELOG_BEGIN
CHANGELOG_END

* fixup ledgerTimeHelper

* Use new ledger time algorithm

* Mark LET/MRT as deprecated

CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
  The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
  the ledger time of transactions is instead set automatically by the ledger API server.
  Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
  ledger time of transactions is greater than or equal to the ledger time of any used contract.
  See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END

* Add ledger time skew check

* Remove command updater

LET/MRT are now deprecated, this class is now useless

* Remove old time model validator

* Switch to new time model check: kvutils

* Switch to new time model check: in-memory ledger

* Switch to new time model check: SqlLedger

* Use initial ledger config

* Ignore user provided LET

* Use TimeProvider in submission services

* Use deduplication_time in daml-script runner

- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.

* Use arbitrary default value for deduplication time

* Use built-in Instant ordering

* Remove obsolete test

* Remove obsolete test: CommandStaticTimeIT

* Refactor test: TransactionMRTCompliance

* Disable test: CommandTrackerFlow timeout

* thread maxDeduplicationTime through to CommandTracker

* Improve test

* Refactor command client configuration

* Deduplication time should always use UTC

* Add missing method in TimedIndexService after rebase

* Put more details into the deduplication error response.

* Use system time for command dedup submittedAt.

* Use explicit UTC time source in command validator

* Revert CommandTracker[Flow] to previous completion-recovering-behavior

* Adapt scala client command config to new config params

Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
2020-03-25 09:28:56 +01:00
Andreas Herrmann
3b550b9933
Avoid recompiling loaded packages in REPL (#5157)
* Use MutableCompiledPackages in REPL

This avoids unnecessary recompilation of loaded packages.

CHANGELOG_BEGIN
CHANGELOG_END

* Track compiled definitions explicitly

Compile packages on load directly.

Use `PureCompiledPackages` extended with package for current line in
REPL service `runScript`.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-24 16:53:40 +00:00
Andreas Herrmann
4f7c52c981
DAML script runner takes CompiledPackages (#5110)
* daml-script runner from compiledPackages

* Factor out script parameter conversion

* Use ADT for DAML script (Action & Function)

* run --> runwithClients

* DAMl script Runner `run` convenience function

* REPL use pure script runner constructor

* DAML script runner holds script itself

* scalafmt

* comments

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-20 17:52:54 +00:00
Andreas Herrmann
8527ec4c4b
Infer daml-script package id from script (#5076)
* Factor out common identifier generation

For `DA.Types`, `DA.Internal.Any`, and `Daml.Script`.

* Factor out Script type for DAML scripts

* Adapt DAML script test runners

* Adapt REPL

CHANGELOG_BEGIN
CHANGELOG_END

* ./fmt.sh

* Avoid `unapply`

addressing
https://github.com/digital-asset/daml/pull/5076#discussion_r394526881

* Pure Script.fromIdentifier

* Pure Script.fromDar

* Simplify test script discovery

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-03-19 11:59:06 +00:00
Moritz Kiefer
44d843f9ef
Support authentication and TLS in DAML repl (#4998)
* Support authentication and TLS in DAML repl

changelog_begin

- [DAML Repl - Experimental] You can now connect to a ledger via TLS
  by passing ``--tls`` to ``daml repl``

- [DAML Repl - Experimental] You can now connect to a ledger with
  authentication by passing the token via ``--access-token-file`` to
  ``daml repl``.

changelog_end

* try to fix linking on windows

* windows is weird

* gnah
2020-03-16 10:43:57 +01:00
Gary Verhaegen
2eedd291ec
bind repl to localhost (#4863)
Currently the repl server is bound to 0.0.0.0, which is not great for
security and makes running the tests a bit disruptive on macOS.

This binds it to 127.0.0.1 instead.

CHANGELOG_BEGIN
- [DAML Repl - Experimental] The REPL server will now bind to 127.0.0.1
  instead of 0.0.0.0.
CHANGELOG_END
2020-03-06 11:40:55 +01:00
Moritz Kiefer
2a05611b63
Graceful error handling in daml repl (#4673)
* Graceful error handling in `daml repl`

This PR changes `daml repl` to handle errors (parse errors, type
errors, unsupported statement errors, script errors) gracefully
and just emit an error message instead of tearing down the whole
process.

This gets the repl into a state where I think it’s sufficiently
user-friendly to be released (obviously there are tons of potential
improvements). The only thing missing before I’m comfortable
mentioning this in release notes and uninternalizing it are docs.
If you think there is something crucial that needs to be addressed
before, let me know.

changelog_begin
changelog_end

* why is windows
2020-02-24 18:15:32 +01:00