Commit Graph

1344 Commits

Author SHA1 Message Date
Remy
21c1c951f1
LF: remove control of input/output value/transaction versions. (#7858)
This is made obsolete by #7788.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-02 20:02:53 +01:00
Martin Huschenbett
6572457973
damlc: Use fresh names when compiling builtins (#7809)
* damlc: Use fresh names when compiling builtins

Currently, we're always using the variables names `v1`, `v2`, ... for
the lambdas we introduce during the conversion of partially applied
builtins to DAML-LF. This can lead to problems when the variable name
is already in use. The test cases used to fail because of the usage of
`v2`.

This PR fixes the issue by properly generating fresh names. There's no
test case for type variables since I don't know how to trigger an
issue. Fixing it seems a good idea nevertheless.

CHANGELOG_BEGIN
CHANGELOG_END

* Add Sofia's suggestions

CHANGELOG_BEGIN
CHANGELOG_END

* Fix a nasty typo

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Fix more issues pointed out by Sofia

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2020-10-29 14:17:52 +01:00
Jussi Mäki
fa9bc4a984
Cache computation of top-level values at definition level (#7818)
* Cache computation of top-level values at definition level

Earlier the computation of a top-level value was only cached at the
use-site in SEVal. This introduces SDefinition which contains the same
mechanism as SEVal to cache the computation.

As expected this does not impact performance much:

before: CollectAuthority.bench  //daml-lf/scenario-interpreter/CollectAuthority.dar  CollectAuthority:test  avgt   40  44.267 ± 0.728  ms/op
after: CollectAuthority.bench  //daml-lf/scenario-interpreter/CollectAuthority.dar  CollectAuthority:test  avgt   40  43.693 ± 0.702  ms/op

What this does have a significant impact is on reducing the number of distinct
SValues for things like type class dictionaries etc, so that now we have one
SValue per dictionary rather than one per SEVal.

CHANGELOG_BEGIN
CHANGELOG_END

* Address code review

* Fix speedy tests
2020-10-28 13:39:20 +00:00
Sofia Faro
5aa5ea6f41
Add roundtrip tests for MetadataEncoding. (#7833)
* Add roundtrip tests for MetadataEncoding.

changelog_begin
changelog_end

* buildifier fixx
2020-10-28 13:09:40 +00:00
Sofia Faro
3f64a6ae8d
Add MINIMAL pragma support in data-deps. (#7815)
* Add MINIMAL pragma support in data-deps.

CHANGELOG_BEGIN

- [DAML Compiler] MINIMAL pragmas are now imported correctly
  in data-dependencies.

CHANGELOG_END

* fix test

* Review comments

* Update compiler/damlc/daml-lf-conversion/src/DA/Daml/LFConversion.hs

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-10-28 10:37:23 +00:00
Sofia Faro
201a6e8c1b
Refactor fun dep and overlap mode implementations. (#7810)
* Refactor fun dep and overlap mode implementations.

This PR moves the functional dependency and overlap
mode encoding/decoding functions to a single module,
separate from the rest of LF conversion or
data-dependencies, in preparation of adding more
annotations of this sort (e.g. MINIMAL).

This involves a small amount of refactoring to get
the types to line up... It should be a little easer
to write some roundtrip tests for these now.

I'm not super satisfied with how much fun dep / overlap
mode logic is still left in LFConversion.hs and
DataDependencies.hs, but this is a step in the right
direction at least.

changelog_begin
changelog_end

* Rename Encoding to MetadatEncoding
2020-10-26 15:59:39 +00:00
Martin Huschenbett
e49b07092a
damlc: Minor cleanup in setFieldPrim conversion (#7802)
That `setFieldPrim` is so far away from `getFieldPrim` has bugged me
for a while now. I _strongly_ feel that they should be next to each
other.

There are also a few outdated comments that are brought up to date.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-26 08:57:25 +00:00
Martin Huschenbett
9dec66899b
Compiler: Add more tests for record projection conversion (#7786)
This PR adds a couple of tests that ensure that various syntactic
variants of record projection all get translated into the according
DAML-LF primitive as soon as we know the record type and field name.

This PR also fixes some inconsistencies in the translations of
`getField` and `getFieldPrim`. The change for `getField` does not have
any impact if the simplifier is run since the simplifier would perform
the inlining for a partially applied `getField` we perform here.

Nevertheless, performing this inlining in the conversion to DAML-LF has
some advantages. First of all, we also run it in cases where the
simplifier is turned off, like incremental computation. Second, this is
such a crucial optimization that I want to be as sure as possible that
it is run. Third, doing the inlining in the conversion is cheap and
removes a bit of load from the simplifier. Fourth, the consistency with
the other record primitives is quite nice too.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-23 13:32:33 +02:00
Martin Huschenbett
35a401f575
Fix scenario service vs struct projections bug (#7778)
* Fix scenario service vs struct projections bug

In a recent change, we have sped up the projection from structural
records by means of an inference in the DAML-LF type checker. This
change was made under the assumption that users of the DAML interpreter
run through package validation before compiling to the internal Speedy
AST. Unfortunately, that assumption was wrong and not covered by our
existing tests.  Manually testing of a new release candidate reminded us
of the fact that the scenario service skips package validation for the
sake of faster response times in the IDE.

This PR drops the assumption that package validation is always run.
Instead, we add the old implementation of struct projection, which works
without DAML-LF type checking, back and use it whenever the AST has not
been annotated with the information inferred by the type checker. We do
the same for struct updates.

We improve test coverage by _additionally_ running the `damlc`
integration tests without package validation for the latest stable
version of DAML-LF and DAML-LF 1.dev. These tests would have caught the
issue we only discovered during manual testing.

CHANGELOG_BEGIN
CHANGELOG_END

* Add some explanations

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-23 13:30:56 +02:00
Robin Krom
cf85751c3c
daml assistant: cleanup of daml ledger commands (#7777)
* daml assistant: cleanup of daml ledger commands

This is a cleanup of the Helper/Ledger.hs module in preparation for
further implementations of `daml ledger` commands against the JSON API.
In particular the module Compiler/Fetch.hs is removed from daml-compiler
and its contents moved over the Helper/Ledger.hs. Some renaming and
reorganization is done for clarity.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-22 17:07:43 +02:00
Martin Huschenbett
51a97d44e4
Hide Symbol kind from stdlib docs (#7752)
`Symbol` is the kind of type-level strings. We use type-level strings
as part of our record system. That's why we need the `Symbol` in the
first place. However, one can abstract over record fields using the
`HasField` typeclass without directly using the `Symbol` kind. Thus,
there's is no need for our users to ever use it. In fact, they
most likely don't want to use it since it does work work with
data-dependencies and we have no plans to support it in that way.

Given that `Symbol` is not necessary and doesn't work in all cases,
let's hide it from the docs so that nobody gets the idea to use it.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-22 14:23:46 +02:00
Bernhard Elsner
e6a276e591
Add Enum and Bounded instances to DA.Date (#7774)
* Add Enum and Bounded instances to DA.Date

CHANGELOG_BEGIN
[DAML StdLib] Add Enum and Bounded instances to DA.Date
CHANGELOG_END

* Hide internal date functions in docs

* Fix line numbers

* Export Date<>Int conversion functions rather than hiding

* Update compiler/damlc/daml-stdlib-src/DA/Date.daml

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Offset the Date enum to make it zero-based

* Apply suggestions from code review

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>

* Improve tests

* Incorporate Martin's feedback

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-10-22 11:46:13 +00:00
Sofia Faro
b925c483d2
Add overlapping instances support in data-deps. (#7770)
Also adds a small test.

changelog_begin

- [DAML Compiler] Similar to functional dependencies, the
  compiler now adds overlapping instance pragmas into the
  DALF/DAR files. This way, overlapping instances pragmas
  will be preserved when importing typeclass instances
  via data-dependencies.

changelog_end
2020-10-21 15:07:10 +01:00
Sofia Faro
1d64fca39a
Add overlap mode annotations during LF conversion. (#7765)
* Add overlap mode annotations during LF conversion.

This PR adds an overlap mode binding `$$om$fFoo...` during LF conversion
per typeclass instance `$fFoo...` that uses an overlap mode pragma. The
idea is that we can then pick these up during data-dependencies
(left to a separate PR).

This PR also adds a test to make sure we're capturing this information
correctly.

changelog_begin
changelog_end

* git shrapnel
2020-10-21 12:53:03 +01:00
nickchapman-da
fd5db0cfd7
choice observers, prep (#7548)
* choice observers, WIP

changelog_begin
changelog_end

fix generator driven test

Node.isReplayedBy, consider observers

add observers to NodeExercise in transaction.proto

add observers to TemplateChoice in Ast.scala

add observers to LF .proto, and Haskell Ast for TemplateChoice

reinstate trailing // for better format

fix validate tests

fix haskell LF decoder when choice-observers field is missing in .proto

fix build

make choice-observers optional in scala AST

make choice-observers optional in Haskell Ast

address comments from Remy and Martin

more review comments

check TransactionVersions.minChoiceObservers in Transaction encode/decode

featureChoiceObservers, and check in haskell type-checker

improve speedy Compiler for empty choice-observers

extend scala LF decoder for optional choice observers

extend scala parser for choices to allow optional choice-observers clause, and test

rename new field in scala Ast -> "choiceObservers"

var rename

extend TypingSpec tests for choice-observers. also add missing negative test for controllers

switch from keyword "ob" to identifier "observers" in scala parser choice syntax

add TODO for featureChoiceObservers to be part of DAML 1.9 (issue 7139)

* replace "NICK" comment markers with "FIXME #7709" comment markers
2020-10-21 11:15:18 +01:00
Martin Huschenbett
03a03957d9
Reverts digital-asset/daml#7725 (#7744)
I tried a compiler with these new warnings on a giant codebase and I
turned out they are _extremely_ noisy. There are basically two things we
want to warn about:

1.  Usage of the promoted kinds and types coming from the `DA.Generics`
    module.

    We already have a warning that importing `DA.Generics` will break
    data-dependencies. Adding a warning to every single use site of its
    exports seems rather excessive and doesn't add any new information.

2.  Usage of the `Symbol` kind and type-level strings.

    The `Symbol` kind seems to work better with data-dependencies than I
    expected although it is erased into kind `*` during conversion to
    DAML-LF. This is for two reasons: (1) The only _legal_ way to
    introduce a type variable of kind `Symbol` is by means of the
    `HasField` typeclass (you can technically get your hands on the
    `Symbol` kind directly, but you need work around not being able to
    import the internal module `GHC.Types`). (2) In data-dependencies,
    we reconstruct type variables of kind `*` not as kind `*` but rather
    ask GHC to infer their kinds (via
    [`UserTyVar`](https://hackage.haskell.org/package/ghc-8.10.1/docs/src/GHC.Hs.Types.html#UserTyVar)).
    This means that every type variable that was originally of kind
    `Symbol` will be reconstructed at that kind again since the
    necessary `HasField` constraint will force GHC to infer it.

    I'm totally aware that it looks a lot like this works by accident
    but I'm not completely opposed to making it a feature after the
    fact. Before we make any decisions in this regard though, we should
    decide on what uses of kind `Symbol` we want to make compatible with
    data-dependencies. For instance, functions of types like

        overField: forall x r a. HasField x r a => (a -> a) -> r -> r

    might be something we want to support. We also need to support
    constraints of the form `HasField "abc" ...` with a concrete
    type-level string everywhere. The current implementation would have
    warned about them. Thus, I see two open questions around the
    `Symbol` kind: (1) Do we want to allow `HasField x r a`, where `r`
    is a type variable and _not_ a type-level string, to be a superclass
    constraint? (2) Do we want to allow the same constraint in the
    context of a typeclass instance?

    One thing we should definitely not allow is using the `Symbol` kind
    directly under its name `GHC.Types.Symbol`, which is always in scope
    for our record preprocessor. I will add an error when users try to
    access that kind in a separate PR.

CHANGELOG_BEGIN
[DAML Compiler] The warning about advanced types combined with type
classes in a way that is supported by data-dependencies has been removed
again.
CHANGELOG_END
2020-10-20 22:15:14 +02:00
Sofia Faro
8e4e0f98c5
Functional dependency support in data-dependencies (#7719)
* Functional dependency support in data-dependencies

(WIP)

* Add test.

changelog_begin

- [DAML Compiler] Functional dependencies are now saved
  by the DAML compiler in the DALF file / DAR package. When
  importing a file with the functional dependency data via
  data-dependencies, these will be picked up automatically.

changelog_end
2020-10-19 16:50:45 +01:00
Sofia Faro
5e0559ce51
Move CallStack, HasCallStack docs to DA.Stack (#7735)
* Move CallStack, HasCallStack docs to DA.Stack

This PR only modifies the docs. In particular, it:

- moves the `CallStack` and `HasCallStack` docs to `DA.Stack`
- gives an example of adding the `HasCallStack` constraint
- hides the rest of `GHC.Stack.Types` from the docs

Rationale: `GHC.Stack.Types` has a lot of `TextLit` stuff that
we don't want people to use directly, and we want people to
import `DA.Stack` instead anyway.

changelog_begin
changelog_end

* commit unsaved changes, doh

* grammar
2020-10-19 13:13:43 +00:00
Martin Huschenbett
5f0acbc21f
Add sed script to help with updating the damlc ITs (#7727)
Updating the expected dignostics for `damlc`'s integration tests can
be quite tedious. This PR adds a `sed` script to simplify it a bit.
See the included readme for details.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-19 14:26:59 +02:00
Martin Huschenbett
7074d92324
Warn about un-upgradability of fancy kinds (#7725)
Haskell allows for promoting types to the kind level with the
`DataKinds` extension. Since DAML-LF has no similar concept (and
frankly speaking should't have one), this does not work well with
data-dependencies. Unfortunately, we need to enable the `DataKinds`
extension by default in `damlc` since our record system and the
`Numeric` type depend on it. Thus, issuing a warning that the
`DataKinds` is not supported with data-dependencies is not an option
and we need more refined warnings.

Most uses of promoted kinds won't even compile to DAML-LF and there's
no need to warn about those since they cause hard failures. However,
we need some limited support for type-level strings and the `Symbol`
kind, for type-level numbers and the `Nat` kind, as well as for some
stuff from `DA.Generics`. Type-level numbers and the `Nat` kind work
reasonable well since we have them in DAML-LF too. The `Symbol` kind,
and the types and kinds from `DA.Generics` are "erased" during
conversion to DAML-LF by replacing them with either a special `Erased`
type or the star kind. Thus, these entities are not restored properly
with data-dependencies. In combination with type classes this leads
easily to instances that for different types in the original code but
are at conflict during interface reconstruction. Long story short,
erased types/kinds and type classes don't work with data-dependencies.

We have a hack to somehow preserve type-level string since we need that
for our record system and the `HasField` type class. I have no desire
to support this hack for arbitrary type classes coming from the user.
Thus, I consider type-level strings to be erased as well here.

This PR adds a warning whenever a user is attempting to define a type
class that involves a kind that will be erased or an instance that
involves a type that will be erased. The warning tells the user that
this does not work with data-dependencies and is hence not upgradable.

There is one caveat: the `HasField` instances produced by the
propressor fall in the category we should warn about as well. Since not
supporting them would make our record system unusable, we need to give
them special treatment in data-dependencies. Thus, warning about them
would be unjustified noise and we explicitly exclude them in the code
producing the warnings.

CHANGELOG_BEGIN
[DAML Compiler] Issue warning when advanced types are with type classes
in a way that is supported by data-dependencies.
CHANGELOG_END
2020-10-19 10:13:51 +02:00
Martin Huschenbett
fd1086bb32
Allow more language extensions wrt data-dependencies (#7728)
Stop emitting warnings about un-upgradability when our users use the
`InstanceSigs` or `MultiWayIf` extensions. These are safe.

Also don't issue warnings for `Cpp`, which can only be used internally.

CHANGELOG_BEGIN
[DAML Compiler] Add `InstanceSigs` and `MultiWayIf` to the list of
language extensions that don't cause problems with data-dependencies.
CHANGELOG_END
2020-10-19 10:02:01 +02:00
Moritz Kiefer
d1a883c86a
Remove warnings on AllowAmbiguousTypes (#7715)
changelog_begin
changelog_end
2020-10-16 14:17:32 +02:00
Sofia Faro
1390703cc9
Use $$ instead of $ for new name prefixes in damlc (#7701)
GHC uses $. To avoid a clash, let's use $$ instead as suggested by Martin.

changelog_begin
changelog_end
2020-10-16 12:13:44 +01:00
Martin Huschenbett
c80a4d2322
Warn when importing DA.Generics (#7705)
* Warn when importing DA.Generics

The module `DA.Generics` is hidden from the docs of `daml-stdlib`
because we consider it experimental. On top of that, the comments at the
top of the module make it clear that this module does _not_ work across
different SDK versions. Nevertheless, let's make it crystal clear that
`DA.Generics` does not work with data-dependencies and will stop you
from upgrading your package.

I've decided against using the a `{-# WARNING ... #-}` annotation
on the `DA.Generics` module declaration since that issues a warning for
every single time an enitity from the module is used rather than once
when importing the module. To me, that's a bit over the top since it is
conceivable that people still want to use the module during some stages
of their development. Another thing we might need to consider soon is a
`-fno-data-dependencies-warnings` flag which suppresses warning related
to data-dependencies. This would be impossible with a
`{-# WARNING ... #-}` annotation.

CHANGELOG_BEGIN
[DAML Stdlib] Issue a warning when importing the `DA.Generics` module,
which does not work with data-dependencies.
CHANGELOG_END

* Fix typo

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-10-16 09:25:19 +00:00
Martin Huschenbett
500bb087b7
Show ranges in a uniform format in damlc ITs (#7708)
Unfortunately, diagnostics use 0-based ranges internally but render
them 1-based when showing them to users. This has caused a lot of
confusion in the `damlc` integration tests because we communicate
with our own devs in a mixture of the user facing 1-based format and
the 0-based internal format. On top of that, we had a bug where the
column numbers were off by 1.

Let's put this to an end and display all ranges in the 1-based format
meant for humans. To make sure our parser and renderers for ranges
are in sync, and not again off by 1, we also add a couple of tests to
check that they roundtrip.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-16 09:43:15 +02:00
Martin Huschenbett
44db5de726
Damlc ITs fix pretty range (#7707)
The new version of `ghcide` fixes a bug in the pretty printer for
diagnostics. So far, the column number for an error/warning you would
be shown by `daml build` was always be one smaller than in DAML
Studio. Now, we show the same locations across the command line tools
and the IDE.

We also bring our test files in sync with the ranges the command line
tools print now.

CHANGELOG_BEGIN
[DAML Compiler] Show the correct column numbers in error locations
produced by command line tools like `daml build`.
CHANGELOG_END
2020-10-16 09:06:37 +02:00
Martin Huschenbett
fb33decf09
Warn on potentially un-upgradable language extensions (#7662)
* Warn on potentially un-upgradable language extensions

We issue a warning whenever a module is compiled with a language
extension enabled for which we are not certain that it works with
data-dependencies. Currently, we consider all extensions except for
the ones enables by default, `ApplicativeDo` and
`PartialTypeSignatures` problematic in this regard. The list of
extensions that work fine with data-dependencies might extend over
time but we need to do further research and add further tests before we
add any extensions.

The warning is always put at the location of the module name regardless
of where the `{-# LANGUAGE ... #-}` pragma is actually used. This is due
to the fact that the `ParsedModule` we get from GHC does not contain
the location information of these pragmas. We should probably improve
this over time but I think it is acceptable for now.

CHANGELOG_BEGIN
[DAML Compiler] Warn when a module uses a language extension that
might not work with data-dependencies.
CHANGELOG_END

* Make the warning less polite :)

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-10-15 17:40:13 +00:00
Martin Huschenbett
ac4fef2d44
Bump the version of ghcide we use (#7694)
* Bump the version of ghcide we use

The new version gives access to the `ParsedModule` instead of only the
`ParsedSource` in the preprocessor. See
https://github.com/digital-asset/daml-ghcide/pull/3
for details.

We also need to fix a few things in `bazel-haskell-deps.bzl` to reflect
that we use our fork of `ghcide`, which lives in
`digital-asset/daml-ghcide`, instead of the old
`digital-asset/ghcide`, which we handed ove to the community, now.

CHANGELOG_BEGIN
CHANGELOG_END

* Update to use fixed ghcide version

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-15 15:11:15 +00:00
Sofia Faro
807a6a3f94
Add a representation of functional dependencies during LF conversion. (#7698)
* Add functional dependency metadata in LF.

This PR adds a new value definition during LF conversion, for type classes with functional dependencies. The binding is a representation of the functional dependency, so we can later pick it up in data-dependencies.

For example, given a typeclass `Foo a b c` with the functional dependency `a -> b, c -> a b` will give rise to a new value definition `$fdFoo` (`$$fdFoo` after name mangling) of type:

```
forall (a : *) (b : *) (c : *).
  { _1: {_1: a} -> {_1: b}
  , _2: {_1: c} -> {_1: a, _2: b}
  }
```

where the `{ ... }` are LF structs. This is a simple encoding of the underlying representations for functional dependencies (which is `[([TyVar], [TyVar])]`).

The PR also adds a test to make sure the functional dependency metadata is exposed, and handles kinds other than kind star correctly. The use of this metadata in data-dependencies is left to a separate PR.

changelog_begin
changelog_end

* Add comment
2020-10-15 14:42:01 +00:00
Remy
9b2a7d59e8
LF: enforce non-empty maintainers in contract key in fetchByKey (#7649)
This fixes a bug in the Speedy interpreter. With this change, the
interpreter crashes if it attempts to fetch using a contract key that
has an empty set of maintainers. This propagates to exerciseByKey as
this command is compiled using fetchByKey speedy built-in.

This is a breaking change approved by @bame-da .

CHANGELOG_BEGIN
    [LF] (Bug fix) enforce non-empty maintainers in contract key during fetchByKey and exericiseByKey.
CHANGELOG_END
2020-10-15 12:22:56 +02:00
Moritz Kiefer
598698abb7
Support multi-party queries in daml script (#7682)
Small comment: This is technically not completely backwards compatible
since it messes with type inference. But to hit that you are doing
something sufficiently advanced that you should be able to add the
type annotation that is required

fixes #7635

changelog_begin

- [DAML Script] `query`, `queryContractId` and `queryContractKey` now
  accepts multiple parties using the `IsParties` abstraction used by
  `signatory`, `observer` and other fields. They will return all
  contracts for which any of the given parties is a stakeholder. Since
  `Party` is an instance of `IsParties`, this is fully backwards
  compatible.

changelog_end
2020-10-14 16:03:48 +02:00
Remy
0bb378690d
LF: Add ExerciseByKey Update (#7663)
CHANGELOG_BEGIN
CHANGELOG_END
2020-10-14 12:06:36 +02:00
Martin Huschenbett
5efc68d1e8
Add more tests for data-dependencies (#7670)
This PR adds a tests to check that the following DAML features,
possibly hidden behind language extensions, work with
`data-dependencies`:

* data constructor operators
* type operators
* partial type signatures
* ambiguous types
* instance signatures
* undecidable instances

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-14 09:43:55 +02:00
Martin Huschenbett
71d32810ec
Improve the naming of things in data-dependencies tests (#7665)
`P1` and `P2` are not particularly descriptive module names. Neither
are `type` and `main` good project names.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-13 11:56:01 +00:00
Martin Huschenbett
a1bb5c6b6a
Refactor packaging test for data-dependencies (#7664)
We have plenty of tests that check that a certain feature can be used
with data-dependencies. Most of these tests have exactly the same
structure and have most likely been copied and pasted. Let's put an end
to this and use a function abstracting this test pattern instead. I'll
add quite a few more tests of this kind, so the gains will be even
bigger than shown here.

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-13 11:26:52 +00:00
Remy
dc34d4f637
LF: add PackageInterface a lightweigh Package (#7577)
CHANGELOG_BEGIN
CHANGELOG_END
2020-10-13 12:41:22 +02:00
Sofia Faro
bda44fb084
Add default method support in data-dependencies. (#7645)
This PR adds default method support in data-dependencies,
including default method signatures (the DefaultSignatures
extension). This adds tests for simple default methods,
and for default methods with signatures.

changelog_begin

- [DAML Compiler] `data-dependencies` now support default
  methods in typeclasses, including default method
  type signatures.

changelog_end
2020-10-12 13:49:54 +01:00
Remy
bebbbc14cd
LF: enforce non-empty maintainers in contract key lookup (#7610)
* LF: enforce non-empty maintainer in contracts key lookup

This fixes a bug in the Speedy interpreter.  With this change, the
interpreter crashes if it attempts to lookup using a contract key that
has an empty set of maintainers.

This is a breaking change approved by Bernhard Elsner.

CHANGELOG_BEGIN
- [LF] (Bug fix) enforce non-empty maintainers in contract key during
  lookupByKey.
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Address Moritz's  review.

* Address Martin's review

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-10-12 14:32:46 +02:00
Sofia Faro
703c86d4fd
Hide RelTime's constructor from the docs. (#7638)
changelog_begin
changelog_end
2020-10-12 11:43:22 +01:00
Remy
ea2a637a11
LF: decouple template from record in the Scala Ast. (#7631)
CHANGELOG_BEGIN
CHANGELOG_END
2020-10-12 11:28:30 +02:00
Stephen Compall
664a0c0076
add Action to high-level trigger updateState (#7621)
* add ActionState to the standard library

* use 1 ActionState, 1 get, 1 put in low-level trigger library

* introduce TriggerStateA for updateState

* fix tests and examples for new updateState signature

CHANGELOG_BEGIN
- [Triggers] The ``updateState`` function now returns a ``TriggerStateA``.  This
  is an action like ``TriggerA``, but doesn't permit emitting commands.  Instead
  of taking the state as an argument and returning a new state, you can
  manipulate the state with ``get``, ``put``, and ``modify``.  Any existing
  ``updateState`` can be ported by replacing ``s -> expr`` in the lambda
  expression with ``-> modify $ \s ->``, and then made to look nicer from there
  as desired.
  See `issue #7621 <https://github.com/digital-asset/daml/pull/7621>`__.
CHANGELOG_END

* some DAML docs for updateState and TriggerStateA
2020-10-09 13:56:24 -04:00
Moritz Kiefer
e09c31899d
Connectify user-facing output (#7624)
Together with #7615, this overs all items in #7612.

changelog_begin
changelog_end
2020-10-09 15:45:02 +02:00
Moritz Kiefer
a37d3045a3
Force a rebuild of DamlcVisualize (#7613)
This seems to have a broken cache and this should force a rebuild.

changelog_begin
changelog_end
2020-10-08 15:56:19 +00:00
Remy
cf89f6a74d
LF: enforce non-empty maintainer in contracts key. (#7597)
This fixes a bug in the Speedy interpreter.  With this change, the
interpreter crashes if it attempts to create a contract containing a 
contract key that has an empty set of maintainers.

This is a breaking change approved by Bernhard Elsner. 

CHANGELOG_BEGIN
- [LF] (Bug fix) enforce non-empty maintainers in contract key during
  contract creation.
CHANGELOG_END
2020-10-08 14:59:41 +02:00
nickchapman-da
be79a93f99
fix comment (#7594)
changelog_begin
changelog_end
2020-10-07 11:18:49 +00:00
Moritz Kiefer
1780f4cd93
Fix qualified name handling in DAML REPL (#7544)
* Fix qualified name handling in DAML REPL

This fixes the issue reported in
https://discuss.daml.com/t/how-to-use-qualified-template-names-in-repl-queries/1329/7

We need to print names qualified otherwise, the type signature for
following lines will be ambiguous even if users qualified the name in
their input. Figuring out the right name to use in qualification is
tricky but Luckily GHC provides this already so we just have to make
sure to plug it into the right places.

changelog_begin

- [DAML REPL] Fix a bug where you got an error about a name being
  ambiguous even if you used a qualified name.

changelog_end

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Repl.hs

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Factor out typechecking of imports

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-10-02 12:16:17 +00:00
Bernhard Elsner
cfed918e40
Fix bifunctor docs (#7552)
* Fix bifunctor docs

- Remove non-implemented Haddock markup
- Fix code blocks
- Fix Lexer to accept `≡`
- Change Haskell REPL `>>>` to DAML REPL `daml>`

CHANGELOG_BEGIN
CHANGELOG_END

* Further improvements
2020-10-02 11:48:03 +00:00
Martin Huschenbett
6ec61bf506
Write proper SDK version in DAR manifest for snapshots (#7546)
* Write proper SDK version in DAR manifest for snapshots

Currently, we don't write the actual SDK version in the DAR manifest but
rather the version that has been sanitized for `ghc-pkg`. For proper
releases, these are identical but for snapshots the latter does not
contain the `-snapshot` part and the commit hash because `ghc-pkg` can
only handle numeric components.

This PR changes this behaviour such that we write the actual
unsanitized SDK version in the DAR manifest.

The implementation might look a bit unintuitive. However, we use the
`PackageSdkVersion` type for exactly two things:

1. Writing it into the manifest.
2. Using it to initialize the package database.

As mentioned above, the former should not be sanitized whereas the
latter should. So far, we've done the sanitization right after reading
the SDK version from `daml.yaml` or `DAML_SDK_VERSION`. Now, we do the
sanitization only in the code concerned with initializing the package
database.

CHANGELOG_BEGIN
[DAML Compiler]
- Bugfix: write the proper SDK version in the DAR manifest for
  snapshot releases instead of a sanitized version
CHANGELOG_END

* Drop SdkVersion.toGhcPkgVersion entirely

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-02 12:59:18 +02:00
Sofia Faro
0382a62965
Fix data-deps dictionary function name mismatches. (#7539)
* Fix data-deps dictionary function name mismatches.

Fixes #7362. Incredibly, it's possible to fix this issue in
data-dependencies by sorting instance names in a funny way.
This is definitely a hack, but it should fix this issue,
which affects overlapping instances as well.

This PR adds a regression test.

changelog_begin
changelog_end

* Fix comments

* apply suggestion
2020-10-01 12:11:10 +00:00
Moritz Kiefer
293e5c55fa
Split DecodeDar into a separate module (#7530)
I originally implemented this for another change where I didn’t need it
in the end. However, it seems like a nice cleanup so spinning it out
into a standalone PR.

changelog_begin
changelog_end
2020-09-30 15:44:22 +02:00
Moritz Kiefer
b9a1905dab
Fix exposed packages in DAML REPL (#7529)
To make `--import` work we also need to expose the
packages. Otherwise, we end up with a ton of confusing errors as soon
as you try anything. Luckily, we can do this fairly cheaply by using
the fact that GHC can expose both unit ids as well as just package
names. Until I realized that I started working on a different approach
of reading the DALFs first to resolve unversioned packages but that
seemed much more messy.

changelog_begin
changelog_end
2020-09-30 13:53:42 +02:00
Tamás Kálcza
9d0206fc44
Added undefined function (#7058)
* Added ??? operator.

CHANGELOG_BEGIN
- Added `(???)` to Prelude.
CHANGELOG_END

* Renamed (???) operator to undefined.
2020-09-30 10:56:28 +02:00
nickchapman-da
689582be6e
Tests for all kinds of Authorization Failure. (#7479)
changelog_begin
changelog_end
2020-09-30 09:01:39 +01:00
Sofia Faro
e9cd92f061
Deprecate the "daml 1.2" version header. (#7513)
* changelog_begin

- [DAML] The "daml 1.2" version header is now deprecated.

changelog_end

* fix some line numbers

* fix some more locations
2020-09-29 13:14:59 +00:00
Moritz Kiefer
00b80b8ea3
Separate off-ledger ond on-ledger speedy (#7501)
* Separate off-ledger ond on-ledger speedy

changelog_begin
changelog_end

* Add OnLedgerBuiltin abstract class

changelog_begin
changelog_end

* fix typo

changelog_begin
changelog_end

* Fix borked rebase

changelog_begin
changelog_end

* Remove unscoped withOnLedger

changelog_begin
changelog_end
2020-09-29 10:05:45 +00:00
Sofia Faro
c5d145358d
Patch ghc to add a daml version header marker. (#7489)
* Update ghc to add a daml version header marker.

(WIP)

changelog_begin
changelog_end

* update stack snapshot

* See what happens with daml-doc

* update patch

* update stack snapshot

* unpin stackage

* Update daml docs tests

* Remove version header ann when parsing module doc

* Update the test

* Set final patch commit SHA.

* update stack snapshot

* unpin stackage (mac/linux)

* lint

* unpin stackage (win)
2020-09-28 17:01:20 +00:00
Sofus Mortensen
14f0a4d58e
Tuple tests - was: Added Functor instance for (,,) for completeness with fst3, curry3, etc (#7442)
* Added Functor instance for (,,) for completeness with fst3, curry3, etc

* added missing unit tests for DA.Tuple

* Reverted Functor instance on (,,)

* fix lint expectation line numbers

* Fix line numbers

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-09-25 10:52:04 +02:00
Bernhard Elsner
331ee44978
Documentation of our support and compatibility framework (#7458)
* Documentation of our support and compatibility framework

CHANGELOG_BEGIN
CHANGELOG_END

* Update docs/source/daml-integration-kit/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/support/component-statuses.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Address feedback

* Update docs/source/support/compatibility.rst

Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>

* Add Deprecations and address feedback

* Fix short title underline

* Apply suggestions from code review

Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>

* Improve sentence on Integration Kit

* Imprive SemVer TLDR

* Uncapitalize release candidate

* Fix release timeline image

* Make the DAML Language Server CLI a Labs feature

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
2020-09-24 14:40:57 +00:00
nickchapman-da
eca7c78540
Tests for interleaved execution and authorization (#7476)
* Add new tests to demonstrate that authorizing checking is interleaved with execution. This work is a follow up to PR #7400, and relates to issue #132

changelog_begin
changelog_end

* naming and other small review comments
2020-09-24 11:38:56 +00:00
Martin Huschenbett
cfa05fa893
Deprecate daml damlc package (#7466)
This command will be deprecated as part of the 1.6 release.
`daml build` should be used instead to build DARs.

CHANGELOG_BEGIN
[damlc]
- Deprecate `daml damlc package` command.
CHANGELOG_END
2020-09-24 10:26:34 +00:00
Sofia Faro
fbb29de4b2
Force newtype constructor names to match type name. (#7467)
* Force newtype ctor name to match type name.

This PR fixes the record constructor check to include newtypes, which are represented as records in DAML-LF. The non-inclusion of all newtypes was an oversight. In addition, we add a regression test.

changelog_begin

- [DAML Compiler] bugfix: Newtype constructors must have the same
  name as the newtype itself, since newtypes are record types.

changelog_end

* Change error message on newtypes

* update a test
2020-09-23 15:36:18 +00:00
Remy
f5694ee2ea
LF: Kill ValueStruct (#7457)
Struct is not serializable and therefore should not appear as Value.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-22 13:37:59 +02:00
Remy
a899e5e4e6
Speedy: Check language version as part of Speedy compilation. (#7440)
Additionnally
+ move allowedLanguageVersion inside compiler config
+ add missing catch error insde
  ConcurrentCompiledPackages#addPackageInternal
+ implement EngineConfig method to easily produce a Compiler.Config

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-22 12:43:48 +02:00
Moritz Kiefer
566d3a21d1
Remove version header from vscode snippet (#7452)
changelog_begin
changelog_end
2020-09-21 16:59:47 +02:00
nickchapman-da
4e32fde001
Cleanup following interleave execution and authorization (#7437)
* remove failedTransactions field from ScenarioLedger.RichTransaction

changelog_begin
changelog_end

* remove Blinding.checkAuthorizationAndBlind; fixup callers to use Blinding.blind when blindningInfo is required

* rename/relocate: ScenarioLedger.CommitError.FailedAuthorizations --> SError.DamlEFailedAuthorization

* fix types to demonstate that at most one FailedAuthorization is detected/reported

* address small review comments
2020-09-18 19:05:19 +01:00
Remy
76ef4c861a
Speedy: group compiler parameters into a Config case class. (#7438)
* Speedy: group compiler parameters into a Config case class.

Speedy compiler is parametrized by several option flags.  Those flags
are passed through different calls (in particular in the instance of
`CompiledPackages`.

This PR group all this configuration parameters into a case class to
pass the option in a cleaner way.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-18 19:32:26 +02:00
nickchapman-da
96d704b25b
Perform authorization checks during execution and not as a separate post-execution phase. (#7400)
changelog_begin
changelog_end

Adapt test for small error message change. An improvement! Previously an internal message was shown. Make test flexible enough to pass for old & new message.

Adapt expected output file. Only change is the contract-ids for active-contracts when the test ends.

fix small test bugs in testcase where authorization is wrong

temp disabled 3 tests; needs investigation; see TODO markers

temp adapt 1 test for change in error message

temp disable 1 test. needs invesigation

undo accidentally commited change to .bazelrc

add copyright header to new file

remove testcase (badActorCheck2) which is no longer expected behaviour, when authorization occurs during execution

address comments: be mre private & other tiny changes

appease scala formatter

improve expected error message in KeyNotVisibleStakeholders testcase

fix authorization issues and re-enable the 3 failing tests in EngineTest which now pass

fix auth issue and re-enable ledger-test-tool test: WronglyTypedContractIdIT

fix compatibility

re-enable test in KVUtilsTransactionSpec.scala
2020-09-17 17:50:04 +01:00
Stephen Compall
a670e68c72
add 'alter' function to DA.Map (#7427)
* add 'alter' function to DA.Map

CHANGELOG_BEGIN
CHANGELOG_END

* swap arguments for alter

- suggested by @cocreature; thanks

* alter moved to stdlib, remove from trigger lib

* use markdown formatting for alter doc

- suggested by @sofiafaro-da; thanks

* fix code block formatting

- suggested by @sofiafaro-da; thanks
2020-09-17 16:46:37 +00:00
Stephen Compall
1926ddbe9d
avoid ACS linear contract ID lookups in Trigger library (#7425)
* add Ord (ContractId a) instance

* change inner list in activeContracts to map

* adapt Internal module to new map

* support rest of Trigger daml lib

* rename adjust to alter, its Data.Map name

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* distribute type arguments for ACS activeContracts

- suggested by @cocreature; thanks

* swap arguments for alter

- suggested by @cocreature; thanks

* reformat groupActiveContracts signature

- suggested by @cocreature; thanks
2020-09-17 15:31:03 +00:00
Stephen Compall
09014dc637
use GenMaps for trigger ACS (#7365)
* use GenMaps for trigger ACS

* Next removed for 1.dev

* temp port trigger test code to 1.dev only

* run trigger tests on 1.dev only

* move pending back to TextMap

* include trigger service in the 1.dev test lineup

- it takes >2min, so shouldn't be permanent

* add Ord TypeRep and Ord TemplateTypeRep when possible

* swap names in Internal to reduce the diff

* try to enable cpp for triggers compilation

$ bazel build //triggers/daml:daml-trigger-1.dev
<snip>
File:     daml/Daml/Trigger/Internal.daml
Hidden:   no
Range:    103:-1-103:-1
Source:   CPP
Severity: DsError
Message:  22 in hpp-0.6.1:Hpp.CmdLine
File:     daml/Daml/Trigger.daml
Hidden:   no
Range:    103:-1-103:-1
Source:   CPP
Severity: DsError
Message:  22 in hpp-0.6.1:Hpp.CmdLine
ERROR: Creation of DAR file failed.
<snip>

* remove problematic options for invoking cpp

hpp: Couldn't open input file: -Werror
CallStack (from HasCallStack):
  error, called at src/Hpp/CmdLine.hs:103:22 in hpp-0.6.1:Hpp.CmdLine

* enough cpp so default and 1.dev triggers compile

* cpp needed for docs as well

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* return trigger service to testing against sdk default lf version

* run trigger integration test against sdk default and LF 1.dev

* return trigger scenario test to SDK default LF version

* avoid import warnings in trigger lib

* Windows manifests a missing file differently. Hilarious
2020-09-15 12:30:23 -04:00
Sofia Faro
ef67efbaaf
Fix missing date under/overflow in DA.Date.date (#7393)
* Fix missing date under/overflow in DA.Date.date

changelog_begin

- [DAML Standard Library] Bugfix: DA.Date.date
  now raises an error when the day argument is
  outside the valid range.

changelog_end

* Add test for underflow too
2020-09-14 16:09:18 +01:00
dependabot[bot]
166c799618
Bump node-fetch from 2.6.0 to 2.6.1 in /compiler/daml-extension (#7389)
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

changelog_begin
changelog_end

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-14 11:44:15 +00:00
Moritz Kiefer
84fe505a58
Add queryContractKey to DAML Script (#7354)
* Add queryContractKey to DAML Script

This matches the behavior and the implementation of
`queryContractId`. We only return contracts for stakeholders and we
return an `Optional` so you can handle lookup failures. On the JSON
API and in DAML Studio this is fairly efficient, over the gRPC API it
degrades to a linear search.

changelog_begin

- [DAML Script] Add `queryContractKey` to the DAML Script API.

changelog_end

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-09-10 19:39:23 +02:00
Moritz Kiefer
c0b6851abe
Generate hoogle docs for daml script/triggers (#7371)
* Generate hoogle docs for daml script/triggers

This PR switches over the documentation generation for daml script and
daml triggers to the multi-page format we already use for the standard
library and extends it to also generate hoogle documentation.

All 3 hoogle files are combined in a single hoogle_db.tar.gz archive.

Since the location in the multi-page format is different, I’ve added
redirects.

I verified locally, that I can generate the hoogle database and that
the links point to the right places.

changelog_begin
changelog_end

* Fix baseurl for daml-stdlib

changelog_begin
changelog_end
2020-09-10 15:58:15 +02:00
Sofia Faro
245f87fcaa
Add warning for variant constructor with unit arg. (#7369)
* Add warning for variant constructor with unit arg.

changelog_begin
- [DAML Compiler] The compiler will now emit a warning when you have a variant type constructor with a single argument of unit type ``()``. For example, ``data Foo = Bar () | Baz`` will result in a warning on the constructor ``Bar``.  This is because the unit type will not be preserved when importing the package via data-dependencies. The correct solution, usually, is to remove the argument from the constructor: ``data Foo = Bar | Baz``. Note that this rule does not apply when the variant type has only one constructor, since no ambiguity arises.
changelog_end

* Rename checks in preprocessor

* Weird lint but ok
2020-09-10 14:27:36 +01:00
Martin Huschenbett
a239edb0eb
damlc: Don't use uniques for type vars during conversion to LF (#7361)
We had some issues with shadowing of type variables in our DAML-LF type
checkers because of some bugs in our implementations of substitution. As
a workaround for these issues, we suffixed each type variable with its
GHC unique during conversion to DAML-LF. Unfortunately, this leads to
new problems when using parametrized variant types and record
projections in combination with data-dependencies.

This PR, drops the unique suffix when converting to DAML-LF. This
change is possible since we fixed the bugs in the substitutions a while
ago. Fortunately, this also fixes the aforementioned problem regarding
data-dependencies.

This fixes #7284.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-09 13:40:28 +00:00
Sofia Faro
220cfc5c79
Disable multipackage LSP tests on Windows. (#7360)
* Disable multi-package lsp-tests on windows.

These are consistently timing out on Windows for unknown reasons.
Disabling until we figure out how to fix the tests. See #4904

changelog_begin
changelog_end

* fix definition
2020-09-09 12:50:30 +00:00
Remy
78e770bf72
LF: Use the new Struct data structure for ValueStruct (#7241)
This PR uses the new data structure introduced in #7220.
Additionnally this fix `Value Equal instance` which was considering
<a: x, b: y> different from <b:y, a:x>.

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-04 18:32:39 +02:00
Moritz Kiefer
3171bb588a
Migrate trigger assertion library to DAML Script (#7318)
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).

changelog_begin

- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.

changelog_end
2020-09-04 13:48:50 +02:00
Moritz Kiefer
0903b1a6be
Filter implicit parameter instances from data-dependencies (#7320)
* Filter implicit parameter instances from data-dependencies

Occasionally GHC likes to float implicit parameters to the top-level
making them look like any other typeclass instances. Implicit
parameters are by design not unique so if we try to reconstruct them
we get an error about a duplicate instance.

I haven’t figured out when exactly GHC floats things to the top-level
but I did verify that this testcase breaks without the fix.

changelog_begin
changelog_end

* Assert that there are 2 implicit instances

changelog_begin
changelog_end
2020-09-04 07:36:37 +00:00
Sofia Faro
a938fd7773
Preserve empty variant constructor in data-deps. (#7303)
* Preserve empty variant constructor in data-deps.

This fixes #7207 and adds a regression test. In a
separate PR I'll add a warning for variants with
single argument of unit type and add a changelog
entry.

changelog_begin
changelog_end

* Add special case for single constructor variants

* Add test for special case
2020-09-02 21:16:16 +01:00
Moritz Kiefer
5e8196061e
Use implode in prettySrcLoc (#7305)
As pointed out by @hurryabit this is a bit nicer. The foldr was just
copied over from GHC.

changelog_begin
changelog_end
2020-09-02 12:43:01 +00:00
Moritz Kiefer
4460cb5e08
Fix trace statement handling in DAML Script service (#7292)
* Fix trace statement handling in DAML Script service

This PR makes sure that we get both trace statements coming from the
ledger client as well as from the ledger server. In the script service
those are two different Speedy machines so we need to interleave
them. This is a bit messy so let me explain the steps (apologies for
not splitting it up but I think it’s easier to understand the
motivation when it is a single PR):

1. Abstract over the tracelog interface. The ringbuffer doesn’t makes
   sense for our purposes and is annoying to clear.

2. Pass in the respective fields of `Machine` to `Conversions` instead
   of the whole machine. This is both a simpler design (passing around
   mutable objects scares me) and it allows us to assemble the fields
   from different machines.

3. Initialize the ledger machine with a simple ArrayBuffer tracelog.

4. After `submit` and `submitMustFail` copy the tracelog from the
   ledger to the client.

fixes #7280

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2020-09-01 17:15:48 +00:00
Moritz Kiefer
09a1b445df
Only include stakeholder contracts in result of query (#7291)
* Only include stakeholder contracts in result of query

This fixes a bug in the script service. We need to filter out divulged
contracts since this should behave exactly like the ACS endpoint on
the ledger API.

changelog_begin
changelog_end

* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/scenario/ScenarioLedger.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* fmt

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-09-01 10:34:11 +00:00
Moritz Kiefer
f8a82ad25f
Add queryContractId to DAML Script (#7289)
* Add queryContractId to DAML Script

I’ve switched between a few different naming options and in the end
settled on this one:

- `lookupContract`, not too bad but misses the connection to `query`
- `queryCid`, we don’t abbreviate this in other places in our API so I
  don’t think we should here.
- `queryContractId`, makes the connection clear and no
  abbreviation. We could also add `queryContractKey` later

changelog_begin

- [DAML Script] Add a `queryContractId` function for querying for a
  contract with the given identifier.

changelog_end

* Fix test

changelog_begin
changelog_end
2020-09-01 10:31:34 +00:00
Jost Berthold
6874f18f77
DEL-8132 extract DAML LF haskell libraries (scripted) (#7246)
An extraction script that makes the LF reading libraries (DAML LF archive decoding to an AST) usable in other Haskell projects.
This script once existed but is not available any more, so I have resurrected the script, adapted to current usage, and upgraded to the newest stackage LTS that can support it. These libraries _could_ be versioned manually to released SDKs and published if desirable (without making any guarantees about the stability, therefore prefixing the version with `0.`.

CHANGELOG_BEGIN
Tool to extract Haskell libraries for DAML LF archives for use in other Haskell projects
CHANGELOG_END
2020-09-01 12:09:26 +10:00
Moritz Kiefer
1eefb4b27d
Fix partial tx and commit locs in the script service (#7281)
It turns out we were too aggressive in resetting the state. We only
want to reset in `submit` on successful commit and in `submitMustFail`
on failed commits. I’ve also split the reset in two parts, one that is
done at the beginning every time and one at the end. I don’t think it
really makes a difference but it matches what the scenario runner does
which at least makes me stop wondering if the difference matters.

fixes #7275 fixes #7276

changelog_begin
changelog_end
2020-08-31 18:00:20 +02:00
Moritz Kiefer
b1530bed7a
Add passTime to DAML Script (#7269)
I’ve also removed the restriction to only support going forward since
scenarios don’t have that restriction and it seems potentially useful
for tests.

factored out from #7264

changelog_begin

- [DAML Script] Add `passTime` helper to advance the time by the
  given interval.

- [DAML Script] In DAML Studio, you can now set the time to the
  past. This is not supported when running against a ledger.

changelog_end
2020-08-31 13:25:09 +02:00
Moritz Kiefer
6743bf5b9b
Add archiveCmd to DAML Script (#7268)
Factored out from #7264 and added a test. Just makes the migration a
tiny bit easier.

changelog_begin

- [DAML Script] Add `archiveCmd cid` as a convenience wrapper around
`exerciseCmd cid Archive`.

changelog_end
2020-08-31 13:24:48 +02:00
Sofia Faro
4c7b07fec6
Add missing orphan superclass references when generating instances via data-dependencies. (#7267)
* Add test for issue 7256

* Fix the issue of missing references

changelog_begin
changelog_end
2020-08-31 11:57:56 +01:00
Andreas Herrmann
d937a5ebad
Track source locations on Daml.Script.submit (#7252)
* Track source locations on Daml.Script.submit

Adds a `HasCallStack` constraint on `HasSubmit.submit` and
`HasSubmit.submitMustfail`.

We need to move the definition of `HasCallStack` to `GHC.Stack.Types` to
avoid a cyclic dependency between `DA.Stack` and `DA.Internal.LF`.
Alternatively, we could inline `HasCallStack` in `DA.Internal.LF` but it
seems preferable to use the alias on the signature of
`submit(MustFail)`.

We need to adapt the type of the builtins for scenario `submit` and
`submitMustfail`.

changelog_begin
changelog_end

* Test-case for scenario/script source locations

* Test second commit with intermediate definition

Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478518834

* Store the full callstack on submit

For forwards compatibility.

Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478520268

* Document unused location definition

Addressing review commit
https://github.com/digital-asset/daml/pull/7252/files#r478522373

* Make user facing `SrcLoc` zero based

Following LSP precedent rather than GHC

Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478529138

* Add comment for backwards compatibility code

Addressing review comment
https://github.com/digital-asset/daml/pull/7252/files#r478529138

* fmt

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-28 12:22:15 +00:00
Andreas Herrmann
b812f09da9
Disable lsp-tests script tests on Windows (#7235)
The similar scenario tests are also disabled on Windows and the script
tests are flaky on Windows due to timeouts.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-26 07:56:12 +00:00
Moritz Kiefer
d33e130f47
Enable DAML Script service by default and document it (#7229)
* Enable DAML Script service by default and document it

fixes #7193

changelog_begin

- [DAML Script] DAML Scripts are now run in DAML Studio just like
  scenarios. The functionality for inspecting the results is
  identical. If you encounter issues, please report them.

changelog_end

* fix double "is identical"

changelog_begin
changelog_end
2020-08-25 19:53:59 +02:00
Moritz Kiefer
3058af8286
Support HasCallStack constraints (#7217)
This is primarily intended to improve error reporting in DAML Script,
in particular, https://github.com/digital-asset/daml/issues/7192 which
is a blocker for replacing scenarios by DAML Script.

However, I think it is also a very reasonable feature in general.

changelog_begin
changelog_end
2020-08-25 08:54:21 +00:00
Gary Verhaegen
f54e193df6
add visibleByKey (#7146)
Adds a Boolean check for the existence of an active contract with a
given key (that is visible with the current set of authorizations), and
explain `lookupByKey` in terms of it to hopefully clarify the relationship
between `lookupByKey` and `fetchByKey`.

Fixes #7143; see that for more information.

CHANGELOG_BEGIN

- [DAML Standard Library] Added a new function `visibleByKey`
  which, given a contract key, returns True if the current context can
  see an active contract with that key.

CHANGELOG_END
2020-08-24 15:04:05 +02:00
Andreas Herrmann
037cf11710
Add :json command to DAML REPL (#7202)
* REPL test `:json` command

changelog_begin
changelog_end

* Implement :json command in DAML REPL

changelog_begin
- [DAML REPL] You can now convert DAML expressions to JSON in the DAML
  REPL using the meta-command ``:json`` for example ``:json [1, 2, 3]``.
changelog_end

* Extend DAML REPL documentation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-21 14:53:06 +00:00
Remy
6371f51930
PartialTransaction: replace nested Eithers by a custom ADT (#7191)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-21 10:16:36 +02:00
Andreas Herrmann
9f2367f6cc
IDE distinguish scripts and scenarios (#7197)
* IDE distinguish scripts and scenarios

The code lens above DAML script definitions now reads `Script results`
instead of `Scenario results`.

This also adds DAML Script test cases to the lsp-tests.

The functionality for testing scenario and script code-lenses in the
`Utils` module is identical. However, this adds dedicated definitions
for scripts to clearly distinguish the scenario and script test-cases.

changelog_begin
changelog_end

* Add sum type to distinguish Scenario and Script

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-20 16:36:23 +00:00
Moritz Kiefer
255832cefc
Fix failed contract lookups in scenario runner (#7190)
lookupContract catches the exception which results in us continuing
and eventually running into a null-pointer exception.

This is another reason why we should enable the NonUnitStatements wart
but I’ll leave that for a separate PR (last I tried the scenario
service resulted in a ton of false positives).

fixes #7185

changelog_begin
changelog_end
2020-08-20 09:49:03 +02:00
Gary Verhaegen
29dfd2d227
fix typo in date docstring (#7187)
As reported by Gyorgy Balazsi on [the forum].

[the forum]: https://discuss.daml.com/t/day-parameter-overflow-for-the-date-function-simple-leap-year-handling/1068/7?u=gary_verhaegen

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-20 09:18:18 +02:00
Andreas Herrmann
a080b47e34
Reset the machine on every submit in IdeClient (#7172)
* Test-case for operation after submitMustFail

Currently this triggers
```
BErrorClient (ClientIOError (GRPCIOBadStatusCode StatusUnknown (StatusDetails {unStatusDetails = ""})))
```

changelog_begin
changelog_end

* Reset the machine after every submit in IdeClient

This ensures that the machine is not stuck in a brocken state after a
`submitMustFail` completed.

changelog_begin
changelog_end

* Machine cleanup in finally block

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-19 10:26:15 +02:00
Moritz Kiefer
0e833c02a4
Enable NonUnitStatements for repl service (#7161)
changelog_begin
changelog_end
2020-08-18 10:58:00 +02:00
moritzkiefer-da
eed3e3c2dc
Require 2 CPUs for repl integration tests (#7158)
We have seen timeouts on port file waits occasionally.

Those timeouts are already very high so reducing load is a more
sensible option than bumping the timeout.

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-08-17 16:49:35 +00:00
Andreas Herrmann
6969aae7d5
Remove duplicated test-case (#7157)
changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-17 15:49:02 +00:00
Moritz Kiefer
d4d0419c64
Use TO_TEXT_CONTRACT_ID in Show instance of ContractId (#7153)
fixes #7114

This PR changes the Show instance of ContractId and flips the switch
on triggers and DAML Script to run in off-ledger mode.

It also adds a test that for DAML Script we actually get back the
correct contract id.

There is a bit of a design decision here in how we want to print
contract ids, so let me list the options I considered. $cid will stand
for the actual cid and all options are wrapped in markdown inline
code.

1. `"$cid"`. Indistinguishable from string. Suggests that there might
be an IsString instance for ContractId.
2. `<$cid>`. Matches the dummy `<contract-id>` but it’s not a dummy so
I don’t think matching that is benefitial.
3. `$cid`. Easy to spot (contract ids start with # and have no
spaces), clearly not a string but might look slightly weird.

changelog_begin

- [DAML Script/DAML Triggers] When using DAML-LF 1.dev, the `Show` instance of `ContractId` will now display the actual contract id instead of a dummy `<contract-id>` value. Note that this only applies to DAML Script and DAML Triggers not to ledger code.

changelog_end
2020-08-17 17:06:24 +02:00
Andreas Herrmann
2903987a61
Handle all scenario results in DAML Script IDE client (#7154)
* Handle all scenario results in DAML Script IDE client

changelog_begin
changelog_end

* Address review comment

Consistent error messages

* DAML Script getTime in choice

* Handle `SResultNeedTime` in DAML Script IdeClient

* fmt

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-17 13:46:41 +00:00
Moritz Kiefer
ab8b418b72
Add TO_TEXT_CONTRACT_ID primitive (#7137)
* Add TO_TEXT_CONTRACT_ID primitive

This is the first part of #7114.

This PR

* Adds the primitive to the protobuf.
* Handles decoding and encoding in Haskell and Scala.
* Handles typechecking in Haskell and Scala.
* Handles speedy compilation and interpretation in Scala.
* Updates the specification.

This PR does not yet change the standard library to make use of this
primitive.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Avoid extra allocation

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-08-17 08:39:16 +00:00
Andreas Herrmann
2df09194d5
allocateParty and listKnownParties in script service (#7141)
* script service party management test-case

* script service: allocateParty and listKnownParties

Behave like sandbox on party hint. If a hint is given then allocate
exactly that party name and fail if it already exists. Otherwise
generate a fresh party name. For easier debugging the fresh name is
based on the display name.

changelog_begin
changelog_end

* Track parties that were not allocated explicitly

E.g. parties generated by `partyFromText`.

* Handle and test party already exists error

* fmt

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-17 10:36:06 +02:00
Alex Mason
ceeec38ef2
Update example for (**) (#7148)
The chosen example happened to work for at least three well known functions with the given arguments, `(+)`, `(*)` and `(**)`. Making the example a little less ambiguous.
2020-08-17 09:36:13 +02:00
Remy
66ae38da6c
LF: control of input value version in the engine (#6828)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-13 21:15:52 +02:00
Remy
a890618782
LF: Drop completely the support for LF 0 (#7128)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-13 19:59:07 +02:00
Moritz Kiefer
00aacf21d7
Support setTime/getTime in DAML Script service (#7120)
fixes #7001

changelog_begin
changelog_end
2020-08-13 12:26:17 +00:00
Andreas Herrmann
15395b31a2
Document non-reproducible outputs (#7115)
* Explain why vsix is non-reproducible

* Explain why frontend bundle is non-reproducible

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-13 09:19:46 +00:00
Andreas Herrmann
f764c2f627
Support submitMustFail in DAML Script Service (#7076)
* submitMustFail test-case

* script compat test for submitMustFail

changelog_begin
changelog_end

* Support submitMustFail in DAML Script Service

Pushes handling of `submitMustFail` into the DAML script interpreter
rather than DAML script itself.

* fmt

* Simplify IdeClient.submitMustFail

* Comment on case we don't expect to hit

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-12 08:54:34 +00:00
Moritz Kiefer
e1236d42ed
Make ApplicationId in DAML Script configurable (#7074)
This is set per participant since it is similar to the token file.

fixes #7029

changelog_begin

- [DAML Script/DAML REPL] You can now configure the application id via
  `--application-id` or the `--participant-config`. This is primarily
  useful if you are working against a ledger with authentication and
  need to match the application id in your token.

changelog_end
2020-08-10 17:24:38 +02:00
Moritz Kiefer
733a43aeea
Move ApplicationId in daml-script to ledger client (#7070)
This makes much more sense since for things like the JSON API and the
script service, we don’t actually have the application id as a
separate parameter.

I’ve also cleaned up all the arbitrary hardcoded application id in
various tests in favor of a DEFAULT_APPLICATION_ID value.

Next step is #7029

changelog_begin
changelog_end
2020-08-10 13:22:51 +02:00
Sofia Faro
a2077e5558
Inline typeclass projections for unknown dictionaries. (#7057)
* Inline tc projections for unknown dictionaries.

changelog_begin
changelog_end

* remove unrelated change

* this is fine

* safety gate unneeded
2020-08-10 10:55:14 +01:00
Remy
17926c5a8a
LF: move language version from modules to packages (#7064)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-10 11:26:58 +02:00
Moritz Kiefer
f897e7a252
Add a Show instance for Ordering (#7068)
Requested in
https://discuss.daml.com/t/why-isnt-ordering-an-instance-of-show-in-daml/1007
Seems like a very reasonable thing to have.

changelog_begin

- [DAML Standard Library] Add a ``Show`` instance for ``Ordering``.

changelog_end
2020-08-10 10:40:09 +02:00
Stephen Compall
96624a7677
use -Ywarn-unused for all Scala code (#6907)
* add -Ywarn-unused to all scalac options

* remove some unused arguments

* remove some unused definitions

* remove some unused variable names

* suppress some unused variable names

* changeExtension doesn't use baseName

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* work around no plugins in scenario interpreter perf tests

* remove many more unused things

* remove more unused things, restore some used things

* remove more unused things, restore a couple signature mistakes

* missed import

* unused argument

* remove more unused loggingContexts

* some unused code in triggers

* some unused code in sandbox and kvutils

* some unused code in repl-service and daml-script

* some unused code in bindings-rxjava tests

* some unused code in triggers runner

* more comments on silent usages

- suggested by @cocreature; thanks

* fix missing reference in TestCommands

* more unused in triggers

* more unused in sandbox

* more unused in daml-script

* more unused in ledger-client tests

* more unused in triggers

* more unused in kvutils

* more unused in daml-script

* more unused in sandbox

* remove unused in ledger-api-test-tool

* suppress final special case for codegen unused warnings

.../com/daml/sample/mymain/ContractIdNT.scala:24: warning: parameter value ev 0 in method ContractIdNT Value is never used
      implicit def `ContractIdNT Value`[a_a1dk](implicit `ev 0`: ` lfdomainapi`.Value[a_a1dk]): ` lfdomainapi`.Value[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                         ^
.../com/daml/sample/mymain/ContractIdNT.scala:41: warning: parameter value eva_a1dk in method ContractIdNT LfEncodable is never used
      implicit def `ContractIdNT LfEncodable`[a_a1dk](implicit eva_a1dk: ` lfdomainapi`.encoding.LfEncodable[a_a1dk]): ` lfdomainapi`.encoding.LfEncodable[_root_.com.daml.sample.MyMain.ContractIdNT[a_a1dk]] = {
                                                               ^

* one more unused in daml-script

* special scaladoc rules may need silencer, too

* unused in compatibility/sandbox-migration

* more commas, a different way to `find`

- suggested by @remyhaemmerle-da; thanks
2020-08-07 13:16:09 -04:00
Remy
8bb4cccdd8
Scenario Service: Use an unique lf version for all the loaed modules. (#7062)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-08-07 16:42:28 +02:00
Andreas Herrmann
322c9eb20b
Support createAndExerciseCmd in DAML Script Service (#7022)
* testCreateAndExercise test-case

* CreateAndExerciseCommand in DAML Script service

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-06 13:44:53 +00:00
Andreas Herrmann
4ced59102b
Support exerciseByKeyCmd in DAML Script Service (#7026)
* Test exerciseByKeyCmd in DAML Script service

* ExerciseByKeyCommand in DAML Script service

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-05 20:24:22 +00:00
Andreas Herrmann
cf24597e70
Factor out reproducibility flags for tar and gzip (#6884)
* Factor out tar/gzip reproducibility flags

* use mktgz in package-app

* Bazel managed tar/gzip

* Remove quiet = True

As stated in the comment this is no longer required with Bazel >= 3.0.

* Build package-app as a sh_binary

This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.

package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.

* Avoid file path too long errors

* Fix readlink -f on MacOS

* Document abspath

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-08-05 14:27:14 +00:00
Moritz Kiefer
ad9d8c24ed
Extend the scenario service with DAML Script support (#6929)
* Extend the scenario service with DAML Script support

This adds most of the infrastructure for running DAML Script via the
scenario service which means it runs as part of DAML Studio and `daml
test`. This is hidden behind a feature flag so we can land this and
parallelize the remaining tasks. The main things that are missing are:

1. `createAndExerciseCmd` and `exerciseByKeyCmd`.
2. Party management needs some work and listing parties is
unsupported.
3. Time management
4. Potentially some better error handling (we need to go through
   SResult and SError and see what is relevant for us).

Overall, it is already in a very usable state and there is a decent
range of tests.

closes #3688

changelog_begin
changelog_end

* Update compiler/damlc/daml-ide-core/src/Development/IDE/Core/Rules/Daml.hs

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* Fix name for actor system and pool

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-08-05 12:56:09 +02:00
Remy
ae7e92a6c4
LF: adapt TransactionBuilder to the new version inference (#6854)
This PR changes TransactionBuilder according the new version inference
algorithm implemented in #6756

CHANGELOG_BEGIN
CHANGELOG_END
2020-08-03 19:05:30 +02:00
Martin Huschenbett
6a678c5733
Add evaluation order tests for foldl and foldr (#6958)
* Add evaluation order tests for foldl and foldr

I'd like to have a play with the implementation of `foldl` and `foldr`
in Speedy in order to improve their performance. Doing so without any
tests for the evaluation order of those builtins, is too scary for me.
Thus, let's kick off the effort by adding some tests.

Except for the usual case where we need to test that we're running over
the list in the right order, we also need to test for the case where
the step function only consumes one argument and the case where the
accumulator is a function. In the latter case we need to make sure we
don't evaluate the argument applied to the final accumulator too early.

We're also very short on the semantics of `foldl` and `foldr` in the
DAML-LF spec but I leave that for another PR.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix copy/paste error

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Minor fixes

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2020-08-03 11:13:50 +00:00
Moritz Kiefer
5ceeb476c8
Remove duplicated tls config parsing from DAML Script/Triggers (#6947)
This was not only unnecessarily duplicated, it also had a bug where
`--crt` behaved like `--pem` instead of setting the cert chain.

I didn’t add new tests since it seems like the wrong place to test
config parsing of a library. We do have tests for TLS in general for
both DAML Script and DAML Triggers.

changelog_begin
changelog_end
2020-07-31 18:25:18 +02:00
Sofia Faro
3b08440fe7
Strip ELocation nodes in simplifier rules. (#6930)
* Strip ELocation nodes in simplifier rules.

Uses stripLoc in the patterns used by the simplifier. This change
reduces the size of the generated code by 1.5% for a very large project,
meaning we had rules (like `let x = e in x`) that weren't trigerring
because of the `ELocation` nodes that weren't being properly ignored.

changelog_begin
changelog_end

* More stripLoc
2020-07-30 14:37:35 +00:00
Martin Huschenbett
4b26a62ad9
Remove some debugging left-overs from damlc tests (#6917)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-29 13:49:16 +00:00
Martin Huschenbett
76e5df6369
Add a test for the conversion of (nested) record updates to DAML-LF (#6915)
* Add a test for the conversion of (nested) record updates to DAML-LF

The new tests check that we generate the DAML-LF we intend to generate,
namely code using `ERecUpd` AST nodes rather than calls to the
`setField` function.

CHANGELOG_BEGIN
CHANGELOG_END

* Use get_value_name instead of get_dotted_name

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-29 13:13:17 +00:00
Martin Huschenbett
f1a83448d5
Mark the damlc integration tests as flaky (#6912)
See https://github.com/digital-asset/daml/issues/6910 for an
explanation of the issue causing the flakiness.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-29 13:10:51 +02:00
Sofia Faro
0f06d63570
Evaluation order specification and tests for exercise and other updates. (#6867)
* Evaluation order for exercise & other updates.

This adds more DAML tests and error cases for, finishing off the update
interpretation for now. I believe this is roughly the limit of what we
can test directly in DAML, and the rest is up to DAML-LF testing.

For example, there's no way to test that the contract inactivity check
in EvUpdExercInactive happens before the evaluation of the controller
expression, since there is no way to generate an 'exercise' with actors
from DAML, as far as I could tell, and 'exercise_without_actor' always
evaluates the controller before passing it off to 'exercise'.

For the same reason, the EvUpdExercBadActor check seems impossible to
trigger from DAML itself. The closest is an authorization test that
happens during submit (which I added a test for, because it is kinda
relevant to evaluation order of scenarios, even though scenario
interpretation isn't really important to the spec).

So at the very least we should add LF tests for those two cases.

changelog_begin
changelog_end

* Apply Moritz\'s suggestions

* Add test with fetch of consumed contract
2020-07-29 09:59:46 +01:00
Martin Huschenbett
6c61ec272e
Add a semantic test for updating a record field twice (#6901)
rhe new test ensures that we don't omit the first update in a record
update that updates the same field twice.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-28 16:14:50 +00:00
Sofia Faro
944ccf9fc5
Expose type synonyms in data-dependencies. (#6894)
* Expose type synonyms in data-dependencies.

This PR converts (non-unit) type synonyms during LF conversion, and then
exposes them via data-dependencies. This is possible only when using an
LF version that supports type synonyms (LF version >= 1.7), the
type synonym isn't of unit type (because it clashes with empty
typeclasses, and the (fully applied) type synonym has kind *
(e.g. you can't define a * -> * synonym in LF).

This fixes issue #6306

changelog_begin

- [DAML Compiler] The DAML compiler will now compile type synonyms
  (``type X = Y``) into the DAR, whenever possible, and will expose
  these synonyms via data-dependencies.

changelog_end

* Add more tests

* Fix formatting
2020-07-28 15:11:17 +01:00
Andreas Herrmann
5481a080d7
DAML REPL :show imports (#6885)
* DAML REPL :show imports

changelog_begin
- [DAML REPL] You can now list the currently imported modules using
  the REPL command ``:show imports``.
changelog_end

* Accept \r\n on Windows.

* REPL functests use text mode file for stdin/out

So that `\r\n` will be mapped to `\n` automatically on Windows.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-28 10:51:48 +02:00
Moritz Kiefer
5668576b78
Upgrade rules-nodejs to the latest release (#6870)
changelog_begin
changelog_end
2020-07-27 16:50:23 +00:00
Andreas Herrmann
1076b165e6
Avoid redundant DAML REPL imports (#6872)
* Deduplicate REPL imports and store in map

changelog_begin
changelog_end

* DAML REPL subsumed imports functest

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-27 14:03:59 +00:00
Andreas Herrmann
e0b9a535b3
DAML REPL :module command (#6865)
* DAML REPL help and module commands

changelog_begin
- [DAML REPL] You can now type ``:help`` at the REPL prompt to see a
  list of possible REPL commands.
- [DAML REPL] You can now add and remove module imports using
  ``:module [+-] Some.Module``.
changelog_end

* REPL :module functests

* Improve module not imported error message

* DAML REPL document REPL commands

* Update docs/source/daml-repl/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-07-27 11:22:29 +00:00
Moritz Kiefer
15bf131fed
Enable assertions in Haskell builds (#6853)
GHC hates its users and defaults to optimizing out assertions. We
fixed that in Buck at some point but clearly that got lost when
migrating to Bazel.

Turns out enabling assertions catches bugs. This insight was brought to
you from the people that also brought you “Turns out writing tests
catches bugs”.

fixes #5624

changelog_begin
changelog_end
2020-07-24 14:29:25 +00:00
Moritz Kiefer
aac32983b7
Support let-bindings in DAML REPL (#6843)
This PR extends DAML REPL to also support `let` bindings which plays
well with the improved support for pure expressions. We support both
pattern bindings and function bindings.

changelog_begin

- [DAML REPL] DAML REPL can now run without a ledger. Take a look at
  the documentation for details.

- [DAML REPL] DAML REPL now supports ``let`` bindings to bind pure
  expressions. Take a look at the documentation for details.

changelog_end
2020-07-23 12:39:51 +00:00
Moritz Kiefer
9881ff5231
Make DAML REPL work without a ledger (#6838)
This PR makes the ``--ledger-host`` and ``--ledger-port`` parameters
optional so DAML REPL works without a ledger which is useful now that
we have better. support for pure expressions. This just piggybacks on
DAML Script’s multiparticipant support so there are no significant
changes on the service.

Docs are updated and we have a testcase.

Side note: What is still missing is `let x = …` in DAML REPL. I’ll
tackle that in a separate PR.

changelog_begin
changelog_end
2020-07-23 10:23:24 +00:00
Andreas Herrmann
4b1438276c
Update Bazel 2.1.0 --> 3.3.1 (#6761)
* Upgrade nixpkgs revision

* Remove unused minio

It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.

* Update Bazel on Windows

changelog_begin
changelog_end

* Fix hlint warnings

The nixpkgs update implied an hlint update which enabled new warnings.

* Fix "Error applying patch"

Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.

* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681

Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.

```
git diff ':(exclude)WORKSPACE'
```

* Update rules_nixpkgs

* nixpkgs location expansion escaping

* Drop --noincompatible_windows_native_test_wrapper

* client_server_test using sh_inline_test

client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.

This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.

* daml_test using sh_inline_test

* daml_doc_test using sh_inline_test

* _daml_validate_test using sh_inline_test

* damlc_compile_test using sh_inline_test

* client_server_test find .exe on Windows

* Bump Windows cache for Bazel update

Remove `clean --expunge` after merge.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-23 09:46:04 +02:00
Moritz Kiefer
5545507738
Improve errors on failing MapKey instances (#6832)
A user hit this recently and the error coming from `fromSome` is
pretty useless.

changelog_begin
changelog_end
2020-07-22 23:32:05 +00:00
Remy
ce3c3c89a4
LF: redesign engine configuration (#6763)
This PR redesigns the engine Configuration as described in #5164. 

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-22 12:17:07 +02:00
Moritz Kiefer
edd84a09d5
Fix reference to return produced by ApplicativeDo (#6821)
* Fix reference to return produced by ApplicativeDo

see https://github.com/digital-asset/ghc/pull/53 for details.

fixes #6820

changelog_begin
changelog_end

* bump to merged commit

changelog_begin
changelog_end

* switch to new ghc-lib

changelog_begin
changelog_end
2020-07-22 10:09:23 +00:00
Sofia Faro
8d5d9090c3
Draft of new operational semantics. (#6598)
* Draft of new operational semantics.

* make errors deterministic, and be explicit about them

* Flesh out the op semantics for expressions and add tests.

* Nail down the semantics of create

* Apply suggestions from code review

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Add a reminder that u has to be a value

... in the update interpretation section.

changelog_begin
changelog_end

* add a test to track the ghc behavior on multi-lets

* add test for erasable type abstraction

* Add non-erasable type abstraction value test.

* move the ValExpTyAbsNat to 1.7-only value test

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-07-21 15:56:44 +00:00
Moritz Kiefer
6df8a98602
Limit JVM memory in a few tests (#6814)
This limits the JVM max memory and initial memory in a few tests that
look like they might be using more than they have two and that run for
a long time so there is a high chance they end up running in parallel
with something else.

changelog_begin
changelog_end
2020-07-21 15:53:36 +00:00
Remy
5468e4d160
DAML-LF: Simplify inference of output transaction/value version. (#6756)
* LF: Simplify inference of output transaction/value version.

As part of #5164, we simplify the way transaction and value versions
are inferred for the engine output.

The versioning of a transaction tr is done as follow:

* Let tvMin be the minimal transaction version allowed by the DAML
  engine configuration
* Let tvMax be the maximal transaction version allowed by the DAML
  engine configuration
* Let n₁, ..., nₘ be the nodes of tr.
* Let pkgᵢ be the package of the template associated to the node nᵢ
* Let lvᵢ be the maximal language version that pkgᵢ uses (directly or
  through its dependencies)
* let tvᵢ be the maximal transaction version supported by all engines
  that support both lvᵢ and tvMin
* Let vvᵢ be the maximal value version supported by all engines that
  support tvᵢ.
* Let tv be the maximal transaction version between tv₁, ..., tvₘ.
* If tv is not greater than tvMax
  + then
    - Version the values of each node nᵢ according vvᵢ
    - Version tr according tv
  + fail otherwise

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-21 17:07:11 +02:00
Moritz Kiefer
6bf1eb19ae
Print results in DAML REPL and support pure expressions (#6799)
* Print results in DAML REPL and support pure expressions

This extends DAML REPL to behave similar to GHCi in that it prints
results if the result is an instance of `Show` and it also supports
pure, non-script expressions now. We copy the same hack from GHCi and
typecheck multiple times. This doesn’t seem to have any noticeable
performance impact (which makes sense, the expressions are tiny and
the overhead of grpc is much much bigger).

Docs and tests are updated.

fixes #6780

changelog_begin
changelog_end

* windows is garbage

* but I fixed it \o/

* Rename getReplLogger to newReplLogger

changelog_begin
changelog_end
2020-07-21 09:01:18 +00:00
Remy
a629959c09
LF: Move GlobalKey and GlobalKeyWithMaintainers outside Node (#6795)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-21 09:08:36 +02:00
Moritz Kiefer
589c240d39
Fix syntax-highlighting of type signatures (#6792)
fixes #6791

This is based on
https://github.com/JustusAdam/language-haskell/blob/master/syntaxes/haskell.YAML-tmLanguage#L1326.

It roughly boils down to the following:

1. Negative look-behind to check that this is not part of something
   else.

2. Match on \\,<-|= (I’ve left out the unicode characters, I don’t want
   to encourage their use in DAML and I’ve also left out ArrowSyntax).

3. Check that the next character doesn’t make this something that is
   part of something else.

As for tests, I’ll refer to https://twitter.com/hillelogram/status/1284189687628824576

changelog_begin
changelog_end
2020-07-20 16:08:23 +02:00
dependabot[bot]
68f46db36e
Bump lodash from 4.17.15 to 4.17.19 in /compiler/daml-extension (#6778)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

changelog_begin
changelog_end

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-17 19:16:20 +00:00
Moritz Kiefer
b12a19181e
Add a --timeout option to daml ledger commands (#6774)
* Add a --timeout option to `daml ledger` commands

At least in the compatibility tests, we see timeouts fairly regularly
in the tests for fetch-dar. If we see those timeouts in the tests, I
see no reason why our users should never see that especially on
actually distributed ledgers and hardcoded timeouts are a bad idea
anyway so now they are configurable.

I’ve also changed the timeout for the fetch tests (those are the only
once I’ve seen timeout) to 120s.

changelog_begin

- [DAML Assistant] The `daml ledger` commands now accept a `--timeout`
  option which can be used to change the timeout for API requests from
  the default of 30s.

changelog_end

* Generate parties as uuids to avoid collisions

changelog_begin
changelog_end

* Match on Deadline Exceeded as well

changelog_begin
changelog_end

* Update compatibility/bazel_tools/test_utils/DA/Test/Util.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Add some docs

changelog_begin
changelog_end

* Better version checks

changelog_begin
changelog_end

* fix imports

changelog_begin
changelog_end

* This is not how you exception

changelog_begin
changelog_end

* Update daml-assistant/daml-helper/src/DA/Daml/Helper/Main.hs

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-07-17 16:35:55 +02:00
Andreas Herrmann
4be01e4df6
Update ghcide 0.1.0 --> 0.2.0 (#6745)
* Update rules_haskell hie-bios support

* Decouple Haskell ghcide and DAML ghcide

Creates a separate `stack_snapshot` to pull in `ghcide` for the Haskell
IDE use case independent of the `ghcide` for DAML. This allows to update
these two `ghcide` instances independently. As DAML uses `ghcide` the
library updates can be involved if the API experienced breaking changes.
At the same time we may wish to update `ghcide` for Haskell earlier to
make use of new features and stay compatible with rules_haskell's ghcide
support.

* Fix Haddock warnings reported by ghcide

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-16 16:30:09 +00:00
Moritz Kiefer
38e1637a6a
Bump timeout of repl tests (#6728)
We’ve added a bunch more tests which now result in us hitting the
timeout sadly.

changelog_begin
changelog_end
2020-07-14 20:37:59 +02:00
Gary Verhaegen
8176fb0c8d
fix the the typo (#6723)
```
s/the the /the /
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 16:55:33 +00:00
Remy
121a6a7d77
DAMLLF: clean up Transaction definitions (#6714)
* LF: move NodeId from value to transaction

* LF: remove some `private` keywords in Transaction

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 14:42:58 +02:00
Andreas Herrmann
ae65f93e01
DAML REPL - Explicit package imports (#6707)
* DAML REPL - Explicit package imports

changelog_begin
- [DAML REPL] The REPL no longer imports all modules from the main DALFs
  of all specified DARs automatically at start-up. Instead, the REPL
  will only import modules from packages specified on the command-line
  using the `--import` flag.
changelog_end

* Accept package-name or package-id

* REPL test case for --import flag

* DAML REPL use `UnitId` for import packages

Addressing review comment
https://github.com/digital-asset/daml/pull/6707#discussion_r453731353

* DAML REPL Parse package-name/id at CLI

* DAML REPL Simplify unversioned pkgs

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-14 11:38:23 +00:00
Moritz Kiefer
741c89784b
Support an empty list of DARs in DAML REPL (#6710)
changelog_begin

- [DAML REPL] The list of DARs passed to DAML REPL can now be empty.

changelog_end
2020-07-13 16:35:50 +00:00
Moritz Kiefer
7421db0afb
Improve documentation on fromSomeNote (#6702)
changelog_begin
changelog_end
2020-07-13 14:03:45 +02:00
Andreas Herrmann
73c6fdffb8
Load multiple DARs into DAML REPL (#6656)
* REPL: Factor out Dar loading

* REPL accept multiple DARs

* REPL CLI take multiple packages

CHANGELOG_BEGIN
- [DAML REPL] The REPL now supports loading multiple DARs specified at
  the command line. The DARs are loaded as data-dependencies, and the
  modules of each DAR's main DALF will be imported into the REPL session
  at startup.
CHANGELOG_END

* REPL test loading multiple DARs

* REPL Load packages from current session

* REPL func-test test symbols from two DARs

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Repl.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-07-09 10:47:21 +00:00
Moritz Kiefer
02c59d4f2a
Revert ANF changes and add a testcase for evaluation order (#6645)
* Revert ANF changes and add a testcase for evaluation order

After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.

Therefore, this PR reverts the following commits:

1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee

Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably

1. https://github.com/digital-asset/daml/pull/6576 which was easy to
   resolve and the change to return SEValue instead of SExpr is still
   nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
   some changes since the constructors changed. If you want to review
   those changes in detail (they are pretty straightforward so not too
   important), it’s probably easiest to check out this PR and run
   ```
   git diff 2cd2a8f2a8
   daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
   ```
   to see the diff to the parent commit of the first commit that
   introduced ANF.

changelog_begin
changelog_end
2020-07-08 12:40:12 +02:00
Andreas Herrmann
1bc119e99a
Update rules_haskell (#6600)
* Update rules_haskell

- Fixes the issue where a `sh_test` wrapping a `haskell_binary` couldn't
  add runfiles as is the case with other Bazel rules.

CHANGELOG_BEGIN
CHANGELOG_END

* Save the runfiles environment at start-up

To work around the fact that `withProgName` overwrites `argv[0]`.
See https://gitlab.haskell.org/ghc/ghc/-/issues/18418.

* damlc_compile_test include damlc runfiles

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-07 09:34:26 +00:00
Samir Talwar
28873913d9
Bazel: Upgrade Buildifier. (#6625)
Buildifier now comes with a handy attachment to catch single `\`
characters inside strings and replace them with `\\` if the escape
sequence is invalid. Skylark/Python will do this at runtime anyway; this
just makes it clearer what the actual behavior is.

I needed to change `\` characters at the end of lines to `\\` manually
in order to stop Buildifier from simply concatenating the lines
together. Everything else was automatic.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-06 16:07:47 +00:00
Sofia Faro
8969b3428f
Fix record update evaluation order. (#6590)
* Fix record update evaluation order.

This PR only fixes the evaluation order for single-constructor record
types. Fixing it for variant records, if we do, will be a major
challenge.

This PR also adds a regression test.

changelog_begin
changelog_end

* fix daml-lf-verify
2020-07-03 11:25:44 +01:00
Gary Verhaegen
2cbd4989f2
hide detailed disclosure information by default (#6587)
As requested by @shaul-da, this PR hides the information added by
@hurryabit in #6571 behind a checkbox, and reverts to plain "X"s by
default (checkbox unchecked).

CHANGELOG_BEGIN
- [DAML Studio] The new S/O/W/D information is hidden behind a top-level
  checkbox (next to Show archived). When that checkbox is not checked
  (which is the default), we display X's as before.
CHANGELOG_END
2020-07-03 12:16:34 +02:00
Moritz Kiefer
817b7ca82c
Fix unmangling of dotted identifiers in scenario service (#6597)
'.' is not a valid character in a mangled name which caused unmangling
to fail. Sadly the scenario service does not properly distinguish
between dotted an undotted names but for now everything we unmangle is
dotted anyway so I’ve taken the easy approach of simply changing our
unmangling to take that into account.

We might want to change the scenario service here but I’ll leave that
for a separate PR.

changelog_begin

- [DAML Studio] Fix a crash in scenarios that referenced records
  originating from definitions like `data T = T1 { f1 : Int } | T2 { f2
  : Int }`.

changelog_end
2020-07-03 11:17:27 +02:00
Martin Huschenbett
6dc2c9c9b8
Only ship disclosure information once in scenario service (#6578)
Currently, the scenario service ships the disclosure information of a
transaction twice. One of the two copies is completely unused. This PR
simply deletes it. I hope this reduces confusion in the future.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-02 13:16:21 +00:00
Martin Huschenbett
4accae15cb
Split damlc integration tests per DAML-LF version (#6577)
* Split damlc integration tests per DAML-LF version

Currently, we have one test for all stable DAML-LF versions. This test
takes quite a while to run and times out occasionally. This PR splits
the big test into one test per stable DAML-LF version.

CHANGELOG_BEGIN
CHANGELOG_END

* Address some feedback
2020-07-02 11:13:33 +00:00
Martin Huschenbett
3296f56519
Distinguish witnesses and divulgences in scenario table view (#6571)
* Distinguish witnesses and divulgences in scenario table view

Currently, we display a "D" (for "Disclosure/Divulgence") in the table
view of the scenario view when a party can see a contract because they
have learnt about it via disclosure.

This change further refines this visibily indicators for disclosure:
We display a "W" for "Witness" (i.e. the party has witnessed the
creation of the contract) and "D" for "Divulgence".

Help for the reviewer: In the scenario service server, witnessing is
called explicit disclosure whereas divulgence is called implicit
disclosure.

CHANGELOG_BEGIN
* [DAML Studio]
  When displaying scenario results in table view in DAML Studio, we now
  indicate _why_ a party knows about the existence of a contract:
  - `S` means the party is a signatory.
  - `O` means the party is an observer.
  - `W` means the party has witnessed the creation of the contract.
  - `D` means the party has learned about the contract via divulgence.
CHANGELOG_END

* Address @cocreature's feedback
2020-07-02 07:58:21 +00:00
Moritz Kiefer
494c661a0a
Delete unused concatEnv from daml-lf-verify (#6572)
Was looking through the code and thought the TODO might hint at a bug
but it turns out this is just unused \o/

changelog_begin
changelog_end
2020-07-02 07:25:04 +00:00
Moritz Kiefer
ef470ee883
Simplify and fix package id handling in daml-lf-verify (#6564)
The whole instPrSelf thing is just repeating what DecodeAsDependency
does. Handling unqualified TypeConNames also sounds very questionable
so I changed that as well. Sadly this doesn’t fix the issue I was
hoping to fix.

changelog_begin
changelog_end
2020-07-02 07:03:43 +00:00
Moritz Kiefer
97aeda8db3
Replace substitutions in daml-lf-verify by DA.Daml.LF.Ast.Subst (#6565)
Implementing substitution once is scary enough, let’s not implement it
twice.

Sadly this is also does not fix the bug I was hoping to address.

I would not be surprised if it does fix some bugs but given that I
don’t know which there is no new testcase.

changelog_begin
changelog_end
2020-07-02 08:34:40 +02:00
Andreas Herrmann
1cdea4ad27
Daml REPL - use data dependencies (#6563)
* Fix haddock typo

* Daml REPL - use data dependencies

Without this a DAR that has a transitive dependency on another DAR
cannot be loaded into the REPL since the interfaces are missing, with
data-dependencies they are reconstructed.

CHANGELOG_BEGIN
- [DAML REPL] The provided DAR is now loaded as a data-dependencies,
  meaning that the REPL supports loading DARs with transitive
  dependencies.
CHANGELOG_END

* REPL test - use indirect DAR dependency

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-07-01 16:11:30 +00:00
nickchapman-da
353d0da6f7
Speedy ANF (#6440)
* ANF transformation in Speedy.

The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.

changelog_begin
changelog_end

* improve comment

* inline functions relocateA/L

* remove comment about scalafmt

* remove commented out alterative def for transformLet1

* improve code by adding incr methods to DepthA/E

* remove (n == 0) special case in trackBindings

* clarify comment further

* improve validate/go to not consume stack for deeply right-nested let-expressions

* address comments from Remy: be private; use final case case; etc

* rename to unsafeCompilationPipeline

* add back some trailing commas

* remove commented-out debug line

* improve comment

* remove dev/debug code in compilationPipeline

* remove commented out code in SEAppGeneral.execute

* undo unrelated code improvement in SValue.scala

* fix compile. object Anf cannot be private
2020-06-30 20:52:44 +00:00
Gert-Jan Bottu
2cd2a8f2a8
Daml lf verification (#6516)
Verification tool update:
- Support for recursion and mutual recursion.
- Significant code cleanup and bugfixes.
- Minor improvements:  keep track of more typing information, handle updates properly when called multiple times, support additional operators, more test cases etc.
2020-06-30 20:40:20 +02:00
Moritz Kiefer
5c0cfcc254
Improve docs for DA.Next.Set.difference (#6544)
changelog_begin
changelog_end
2020-06-30 17:28:47 +02:00
Moritz Kiefer
6213f560ef
Disable scenario service during packaging (#6543)
For `damlc test` and `damlc ide`, `optScenarioService` is (correctly)
enabled. However, every call to `withDamlIdeState` will spawn a
scenario service. This means that during package initialization we
will spawn at least one extra scenario service (potentially) more. On
a simple project this gets `damlc test` from 10s down to 7s and I see
a noticeable (although hard to measure) speedup in IDE startup time.

changelog_begin
changelog_end
2020-06-30 16:45:52 +02:00
Moritz Kiefer
470d4f28a5
Fix damlc test with relative --project-root (#6541)
Returning the original relative path from withProjectRoot makes no
sense since we change the directory in there so this PR fixes this to
return the canonicalized project root.

fixes #6245

changelog_begin

- [DAML Compiler] damlc test --project-root now works with relative
  paths as well.

changelog_end
2020-06-30 16:45:28 +02:00
Sofia Faro
f812aaf8e0
Add some tests for constant lifting. (#6519)
* add simplifier tests

* add some lambda constant lifting tests

changelog_begin
changelog_end

* update copyright

* clarify that the subexpression is a lambda

* Explicit export list in tests module

* don't use lambda in test names :-(
2020-06-29 17:25:38 +01:00
Remy
46b68bd92a
ledger test tool: refactor conformance test dars (#6506)
* ledger test tool: refactor conformance test dars

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-26 13:39:00 +02:00
Remy
6d27825acb
Engine: introduce config to control value/transaction output version (#6428)
* Engine: introduce configuration to control value/transaction output version

This advances the state of #5164

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-26 13:38:38 +02:00
Andreas Herrmann
a51c4d1d2e
DAML REPL support static-time mode (#6485)
* DAML REPL --static-time regression test

* damlc repl --static-time|--wall-clock-time flags

CHANGELOG_BEGIN
- [DAML REPL] The time mode can now be specified using the
  ``--static-time`` and ``--wall-clock-time`` flags.
CHANGELOG_END

* Update compiler/damlc/lib/DA/Cli/Damlc.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Verify the effect of setTime using getTime

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-06-25 13:08:10 +00:00
Martin Huschenbett
c4cd51389e
Display why a party knows about a contract in table view (#6475)
* Display why a party knows about a contract in table view

When displaying scenario results in table view in DAML Studio, we now
indicate _why_ a party knows about the existence of a contract:
- `S` means the party is a signatory.
- `O` means the party is an observer.
- `D` means the party has learned about the contract via disclosure or
  divulgence.

With the information we get from the scenario service there is no
straightforward way of distinguishing between disclosed (you
witnessed the `create`) and divulged (you witnessed a `fetch`)
contracts. Fortunately, both words start with a "D" and we're fine. :)

This addresses the first point of
https://github.com/digital-asset/daml/issues/6412 for the table view.

CHANGELOG_BEGIN
* [DAML Studio]
  When displaying scenario results in table view in DAML Studio, we now
  indicate _why_ a party knows about the existence of a contract:
  - `S` means the party is a signatory.
  - `O` means the party is an observer.
  - `D` means the party has learned about the contract via disclosure or
    divulgence.
CHANGELOG_END

* Add tooltips

CHANGELOG_BEGIN
CHANGELOG_END

* Add test

CHANGELOG_BEGIN
CHANGELOG_END

* Remove tooltip for invisible contracts

CHANGELOG_BEGIN
CHANGELOG_END

* Move parties to then end

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-24 14:55:26 +00:00
Moritz Kiefer
97c74d4a29
Support multiple auth tokens in DAML Script (#6473)
* Support multiple auth tokens in DAML Script

This piggy backs on top of the already existing --participant-config
feature. While you can argue that it might be slightly confusing that
you have to specify the same participant twice to specify different
auth tokens, I think this actually makes sense: In an ideal
world (ignoring any performance issues) you have one participant per
party anyway and one connection per participant specified in the
config file still seems like a very reasonable model.

changelog_begin

- [DAML Script] You can now use DAML Script with multiple auth
  tokens. This is particularly useful if you are working with the JSON
  API where you can only have one party per token or with an IAM that
  only provides single-party tokens. The tokens are specified in the
  participant configuration passed via `--participant-config` in a new
  ``access_token`` field. The existing `--acess-token-file` flag is still supported if you want to use the same token for all connections. See
  https://docs.daml.com/daml-script/index.html#running-daml-script-against-authenticated-ledgers
  for more details.

changelog_end

* I will never understand rst

changelog_begin
changelog_end
2020-06-24 14:43:29 +02:00
Shayne Fletcher
4f68cfc480
strengthen unused * testing (#6467)
changelog_begin
changelog_end
2020-06-23 12:44:11 -04:00
Moritz Kiefer
75eb554841
Prefix IDE include paths with project root (#6463)
* Prefix IDE include paths with project root

As described in #6174, `--include` is broken pretty badly in the IDE
atm. LSP works based on absolute file paths so if you have a relative
include dir you run into two issues:

1. You end up with two GHC sessions e.g, one for `/multidir` and one
   for `.`. That results in fun type errors like “Couldn’t match expected
   type `Text` with actual type `Text`”.
2. The same file can end up being represented twice. Apart from being
   horribly inefficient, this breaks as soon as we try to build a DALF
   since the function for constructing that (correctly) explodes when
   there are two files with the same module name.

This change does not break `daml build` since the project root is
relative there.

fixes #6174

changelog_begin

- [DAML Studio] Fix an issue where use of the `--include` option
  resulted in various confusing type errors. See
  https://github.com/digital-asset/daml/issues/6174

changelog_end

* Stop wasting my time hlint

changelog_begin
changelog_end
2020-06-23 16:58:14 +02:00
Martin Huschenbett
5e97019894
damlc tests: Add suppressed test in Iou12 back (#6446)
The suppressed test works just fine as is. This must be something
pre-historic.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-22 10:57:15 +00:00
Moritz Kiefer
cfb4c182a5
Consistently display stakeholders for key visibility errors (#6434)
* Consistently display stakeholders for key visibility errors

fixes #6404

As pointed out by Bernhard in #6404, the previous behavior was pretty
weird. If the committer was only a divulgee, we only displayed
stakeholders. If the committer was neither a stakeholder nor a
divulgee, we displayed stakeholders + parties the contract has been
divulged to. Given that only stakeholders can do lookups it makes much
more sense to display them consistently which is what this PR
achieves. I’ve also renamed “disclosed to” to “stakeholders” to make
it very explicit what is shown there.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>

* fmt

changelog_begin
changelog_end

* lalala

changelog_begin
changelog_end

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-06-22 10:22:37 +00:00
Moritz Kiefer
038388cdb7
Fix handling of non-visibile contract keys in scenario runner (#6433)
fixes #6403

I am not entirely sure why I thought that using `missingWith` makes
sense here but it clearly doesn’t make sense and resulted in a pretty
bad bug where a transaction both succeeded via `submit` as well as
failed via `submitMustFail` which is clearly the wrong thing to do.

This PR fixes this issue and introduces a `notVisibleWith` function
that does the right thing. I’ve also added some comments and an extra
assertion to clarify things a bit.

changelog_begin
changelog_end
2020-06-22 10:20:24 +02:00
Shayne Fletcher
333bf0471d
Support external anchors (#6386)
* support external anchors

changelog_begin
- `daml docs` now supports an `--input-anchor` argument specifying the
path to a database of external anchors
changelog_end

* Add daml-base-anchors.json to the damlc-dist target
2020-06-21 19:44:36 -04:00
Shayne Fletcher
cec2693dc7
enable -Wunused-matches (#6423)
changelog_begin
changelog_end
2020-06-19 19:35:10 +00:00
Moritz Kiefer
8ee7de78ef
Only print diagnostics from integration tests on failures (#6436)
This makes the tests super noisy and makes it hard to spot the
interesting part.

changelog_begin
changelog_end
2020-06-19 18:13:21 +02:00
Sofia Faro
4d698fc5ed
constant lifting: adjust what gets lifted (#6430)
* Adjust what gets lifted

* dont lift typerep

changelog_begin
changelog_end
2020-06-19 17:09:52 +01:00
Gert-Jan Bottu
e5c62ecce2
Verification tool bugfix for PR#6101 (#6431)
CHANGELOG_BEGIN
- Bugfix: forward references are inlined in create updates.
CHANGELOG_END
2020-06-19 15:04:59 +02:00
Remy
1b1b4eab2c
Speedy: clean machine builder name (#6427)
* Address comment martin made in #6368

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-19 14:58:13 +02:00
Sofia Faro
f5613785e0
Constant lifting transformation in the LF simplifier. (#6101)
* First draft of constant lifting

changelog_begin
changelog_end

* refactoring

* doing stuff

* run simplifier on template exprs

* remove merge artifact

* Fix ExerciseWithoutActors

* add comments

* fix trace order test

* prefix generated val names with their provenance

* Verification tool bugfix

During the value collection phase, when encountering a record projection on a (yet) undefined value, stop searching this branch instead of throwing an error.

* Bump pattern-match-perf memory limit with cocreature`s blessing

* bump again

* Filter generated identifiers from daml script test runner

changelog_begin
changelog_end

* Fix party literals

* Remove inlineClosedExpr for now.

* Improve comments

* Reset script test locations

* Unhashmap

* disable daml-lf-verify quickstart tests for now

Co-authored-by: Gert-Jan Bottu <gertjan.bottu@kuleuven.be>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-06-18 17:36:28 +00:00
Moritz Kiefer
0a8c1eba4e
Integrate time service into DAML Script (#6417)
This integrates the time service into DAML script thereby covering the
main piece of scenarios that was missing from DAML script.

This PR does two things (they are very related and doing them together
makes it much easier to test):

1. It “fixes” `getTime` to return the ledger time in static mode by
   querying the ledger time service instead of defaulting to the Unix
   epoch which is pretty useless and I would consider the old behavior
   a bug. We keep the old behavior via the JSON API since there is no
   time service.

2. It adds `setTime` to set the ledger time via the time service. This
   is only supported in static time mode (sadbonx and other ledgers do
   not expose the time service in wallclock mode because changing time
   makes it not wallclock) or via the JSON API (no time service).

fixes #6220

changelog_begin

- [DAML Script] DAML Script’s `getTime` now correctly handles time
  changes in static time mode and returns the current time by querying
  the time service rather than defaulting to the Unix epoch. Note that
  when run via the JSON API, it still returns the Unix epoch.

- [DAML Script] Add `setTime` to DAML Script which sets the ledger
  time via the ledger API time service. Note that this is only
  supported when running over gRPC in static time mode.

changelog_end
2020-06-18 18:25:24 +02:00
Remy
5a9e7ebc7c
Speedy: refactor Machine builders (#6368)
In this PR we cleanup the constructor for the speedy Machine.

* We remove the `case`  keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
  + one generic builder `Machine.apply`,
  + scenario specific builder,

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-18 15:39:55 +02:00