Commit Graph

434 Commits

Author SHA1 Message Date
associahedron
aeaf3ceab7
Export typeclasses in data-dependencies. (#4247)
* Typeclass declarations

* Add modrefs for typesyns

changelog_begin
changelog_end

* Convert constraints and expose methods using them
2020-01-28 16:44:54 +00:00
Martin Huschenbett
d568c113df
damlc: Allow all build options for damlc ide (#4213)
Currently, the DAML language server use for DAML Studio crashes when the
`build-options` section of `daml.yaml` contains flags like `--package`.
This PR fixes this issue by allowing most of the flags for `damlc build`
for `damlc ide` as well.

CHANGELOG_BEGIN

- [DAML Studio] Allow all command line flags for ``daml build`` for
  DAML Studio as well.

CHANGELOG_END
2020-01-28 17:22:02 +01:00
Andreas Herrmann
74984559e8 Test triggers in scenarios (#4233)
* Bazel test for trigger scenario

* daml-triggers: Allow testing trigger rules in scenarios

CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END

* Allow assertions on create commands

CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END

* Add convenience to construct ACS for testRule

* Add assertions for exercise and exerciseByKey

* fix assert message

* Test assertExercise(ByKey)Cmd

* unpackCommands --> flattenCommands

* Add API documentation

* Document that command ids start from "0"

* generalise command assertions to CanAbortm

* export ACSBuilder type

* Better haddocs for trigger command assertions

* explicit let

* ./fmt.sh

* Fix runfiles on Windows

* Add reference to Bazel issue

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-28 11:50:46 +00:00
Moritz Kiefer
54ace2ce06 Initialize package db in damlc test (#4240)
fixes #3436

changelog_begin

- [DAML Compiler] ``damlc test`` now initializes the packagedb automatically which
  means that it will work on projects that declare custom
  ``dependencies`` in ``daml.yaml`` without having to call ``damlc
  init`` first.

changelog_end
2020-01-28 11:22:25 +00:00
associahedron
02b531d254 Preserve class method names during LF conversion. (#4230)
* Preserve class method names during LF conversion.

This PR changes the LF conversion to pick names for dictionary
fields that will be easier to reconstruct as typeclasses
later.

For superclasses this uses "s_1", "s_2", "s_3" and so on,
and for methods this uses "m_foo", "m_bar", "m_baz" and
so on. (Is this enough C++?)

In the future we might want to distinguish between methods
that are mandatory and methods that are optional ... but
I think this should be good enough for now.

This PR also adds a test.

Fixes #4216

changelog_begin
changelog_end

* lint

* Run test in LF 1.7 as well

* Add test for superclass field.
2020-01-28 10:55:02 +00:00
Moritz Kiefer
8a67aff8de Remove position-mapping tests (#4238)
They have been migrated to `ghcide` in
https://github.com/digital-asset/ghcide/pull/388 which is a much more
sensible place for them to live given that the code is in that repo.

changelog_begin
changelog_end
2020-01-28 10:08:15 +00:00
Moritz Kiefer
3245a7b587 Fail when a data-dependency fails to compile (#4219)
* Fail when a data-dependency fails to compile

Previously, we emitted errors but just kept on going which sometimes
kinda works but is certainly not the right thing to do.

Now we just crash and abort compilation.

changelog_begin
changelog_end

* why is windows

* undo crap
2020-01-27 15:25:47 +00:00
Moritz Kiefer
40292c2b6d
Avoid including daml-stdlib and daml-prim twice (#4222)
This fixes #4114 and cleans up the situation around
`data-dependencies` and `dependencies` as described in #4218.

There is still more work to be done here mostly around ironing out all
the edge cases and producing useful error messages instead of
silentely doing the wrong thing but I’ll leave that to a separate PR.

To test this, I’ve fixed the packaging tests to no longer deduplicate
package ids (which means they would return the wrong number if
daml-prim ends up in there twice) and I addded a test where we have 3
projects:

- `lib`
- `a` which depends on `lib`
- `b` which depends on `lib` via `dependencies` and 'a' via
  `data-dependencies`.

changelog_begin
changelog_end
2020-01-27 14:56:12 +01:00
associahedron
830c2c65f5 Add missing CurrentSdk prefixes in data-dependencies (#4220)
* Expose scenarios in data-dependencies.

Also add some type signatures.

changelog_begin
changelog_end

* Add missing prefixes in data-dependencies
2020-01-27 12:57:35 +01:00
associahedron
3c93b5e6c9
Converting functions in data dependencies. (#4182)
* Converting functions in data dependencies.

changelog_begin
changelog_end

* Add NoOverloadedStrings to the set of extensions in generated packages.

* Just use the fn as its own right hand side

* Restore order of package map (not that it makes a difference)

* Adjust imports

* Weird lint but ok

* Make the test pass somehow

* Dont preprocess enums in GHC.Prim

* Preprocess enums everywhere, and add mod ref as needed.

* Revert preprocessor changes

* Dont expose old-style typeclasses

* Dont convert newstyle typeclasses temporarily either

* Add test for function importing
2020-01-27 10:05:00 +00:00
Moritz Kiefer
3496ce03a4 Convert type classes to LF type synonyms (#4023)
changelog_begin
changelog_end
2020-01-24 09:24:45 +00:00
Moritz Kiefer
93d7b1a472 Bump ghcide to fix GHC panics in LSP tests (#4176)
fixes #4152

changelog_begin
changelog_end
2020-01-24 08:40:57 +00:00
nickchapman-da
62d592ec62
support type synonyms in scala (#4101)
* Support DAML-LF type synonyms in scala.

CHANGELOG_BEGIN
CHANGELOG_END

* dont create synonymns in GenerateSimpleDalf

* extend DAML-LF parser to support type synonyms

* test: expand type synonyms correctly
2020-01-23 10:49:28 +00:00
Shayne Fletcher
e9d67ba9f6
Disambiguate consuming vs pre-consuming choices (#4164)
changelog_begin
- [DAML Compiler] Choices marked explicitly as `preconsuming` are now equivalent to a non-consuming choice that calles `archive self` at the beginning.
changelog_end

Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
2020-01-22 14:24:00 -05:00
Moritz Kiefer
2fb107b505 Remove damlc.jar (#4157)
* Remove damlc.jar

We never advertised or published this externally and our only internal
user has moved off this months ago already.

changelog_begin
changelog_end

* Remove dependency from navigator test lib on damlc jar
2020-01-22 16:44:27 +00:00
Shayne Fletcher
674eaffc70
Depend on ghc-lib-parser-ex (for hlint) (#4109)
changelog_begin
changelog_end

Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
2020-01-22 06:25:51 -05:00
Moritz Kiefer
19dd7ed7c4 Handle unserializable scenario result types (#4156)
* Handle unserializable scenario result types

Previously, the conversions made some attempt at guarding against this
by matching on SPap and PClosure. While it would be possible to extend
this to match on STypeRep, this doesn’t actually fix the issue since
this can be nested, e.g., you can have a record with a field that is
an SPAP.

This PR changes this to simply catch any errors thrown from
`toValue`. While this feels a bit ugly, I think it’s a reasonable fix
for now.

changelog_begin

- [DAML Studio] Scenarios with unserializable result types no longer
  crash the scenario service.

changelog_end

* Only run test on DAML-LF >= 1.7
2020-01-22 11:15:46 +00:00
Moritz Kiefer
8d7c021001 Remove intermediate PackageImport type (#4142)
* Remove intermediate PackageImport type

We can just reuse GHC’s PackageFlag directly which simplifies things a
bit.

changelog_begin
changelog_end

* windows is bad

* meh

* foobar

* why
2020-01-21 20:27:26 +00:00
Moritz Kiefer
359e5c350c
Fix enums in data-dependencies (#4143)
To differentiate between "data X = X" which is translated to a DAML-LF
enum and "data X = X {}" which is translated to a DAML-LF record, we
add a datatype context with GHC.Types.DamlEnum. For
`data-dependencies` this needs to point to Currentsdk.GHC.Types.

changelog_begin
changelog_end
2020-01-21 16:11:31 +01:00
Moritz Kiefer
ca3da8ac7c
Use GHC’s syntax for the --package flag (#4136)
* Use GHC’s syntax for the --package flag

The previous syntax was just based on the Read instance of a 3-tuple
which was pretty much unusable. This changes it to GHC’s much nicer
syntax.

I deliberately did not add a changelog entry for this, since this
flag isn’t something that we have documented at all and it is
currently only useful in combination with data-dependencies.

fixes #4126

changelog_begin
changelog_end

* Fix hlint

* get it to compile 😇

* shut up hlint
2020-01-21 13:53:54 +01:00
Moritz Kiefer
dc3953e52f
Remove the key field from pkg.conf files (#4127)
* Remove the key field from pkg.conf files

We never use this and it only a legacy field that is deprecated in
favor of id in GHC.

There is also a minor change in the docs around ExposePackage which
should at most affect error messages.

changelog_begin
changelog_end

* Better error messages
2020-01-21 13:30:38 +01:00
Moritz Kiefer
7525e73134 Bump timeout of packaging tests (#4137)
The packaging tests have grown a fair bit recently and seem to hit
timeouts sometimes. We might want to split it up into multiple tests
suites but for now this should help.

changelog_begin
changelog_end
2020-01-21 11:18:09 +00:00
Moritz Kiefer
7f4061d55c
Remove code for generation of Template and Choice instances (#4131)
The current code has gotten out of sync with the changes to template
desugaring and is therefore simply broken. In addition to that, we
don’t need this once we can use typeclass instances in combination
with data dependencies.

Initially, we just disabled this instead of removing it since we
thought it might still be useful for older packages before typeclasses
are desugared to type synonyms. But we already have one way of
handling those as evidenced in the tests for the DAVL and once we can
call functions from older packages (which works regardless of whether
they have been built before or after the changes to how typeclasses
are desugared), we have another one so I don’t think it makes any
sense to keep this code around.

changelog_begin
changelog_end
2020-01-21 11:57:08 +01:00
associahedron
d9220c6819 damldocs: Refactoring DA.Daml.Doc.Extract (#4112)
* Refactor damldocs

* Refactor damldocs.

changelog_begin
changelog_end

* More refactoring

* Update copyright headers.

* Add export list for Templates

* Add export list to .Util

* added a little ^
2020-01-20 16:17:25 +00:00
Moritz Kiefer
4804fca5d5 Test data-dependencies for more LF version combinations (#4113)
* Test data-dependencies for more LF version combinations

This PR extends the cross-LF version tests to run for all
combinations where depLfVer >= targetLfVer.

changelog_begin
changelog_end

* Fix comments and add TODO
2020-01-20 14:59:43 +00:00
Moritz Kiefer
67f028b0e8
Support cross-SDK data-dependencies against DAVL (#4107)
This PR fixes a minor issue where we were including more references
than we need which causes problems on older SDKs where GHC.Prim is not
a stable module since GHC.Prim cannot be imported (it’s builtin GHC
magic).

More importantly, this PR adds a test for cross-SDK data-dependencies
against the DAVL DAR which is built with SDK 0.13.32 before we started
making all the changes. The test also includes a slightly hacky but
mostly reasonable way of actually using the templates instead of just
the data types from the DAVL DAR which and a scenario to test this.

changelog_begin
changelog_end
2020-01-20 14:54:02 +01:00
associahedron
4a25c8c166
damldocs: Add a --drop-orphan-instances flag (#4100)
* damldocs: Add a --drop-orphan-instances flag

CHANGELOG_BEGIN

- [DAML SDK] Added a ``--drop-orphan-instances`` flag in ``daml damlc
docs``.

CHANGELOG_END

* Update copyright headers

* Document MOVE annotation better and improve orphan instance filter
2020-01-20 10:15:20 +00:00
Moritz Kiefer
9ece831966
Make DAR generation deterministic (#4104)
This fixes the ZIP modification times in all DARs to a specific
value (1980-01-01) whereas they used the current time before. This
both gives us the nice property that not only our DALF builds but also
DAR builds should be deterministic (and there is a test for this). I
have a suspicion that this could help significantly with build times
and avoid rerunning half of the Scala tests on a change to damlc that
should not change the DALFs.

changelog_begin

- [DAML Compiler] The modification times in a DAR are now fixed to a
  given value which makes the output of ``daml build`` deterministic
  in single-threaded mode (which is the default).

changelog_end
2020-01-20 11:00:10 +01:00
Moritz Kiefer
fa8a92f772
Make data-dependencies work with packages produced by damlc (#4099)
This PR fixes a whole bunch of bugs when using data-dependencies on a
package produced by damlc rather than the handcrafted simple-dalf.

The end result is that we are able to build a package with
`--target=1.6` and use it as a data-dependency from a package with
`-target=1.7`. While the test that I’ve added for this is not
cross-SDK it is arguably even trickier since we cannot rely on the
unit id of daml-stdlib changing the version in the unit id.

The changes all fall into one of the following two categories:

- Special handling for daml-prim and daml-stdlib to avoid collisions
  with the one from the current SDK.
- Special treatment of stable packages to avoid generating new code
  for those.
- Remove a couple of filters for daml-prim since we do need to include
  daml-prim from a data-dependency if it is a different version.

This is definitely not perfect and there are a bunch of rather hacky
parts in here but it seems like a good intermediate step (and it has a
test :))

changelog_begin
changelog_end
2020-01-17 20:09:30 +01:00
Moritz Kiefer
f77e326e0b Generate stable DA.Time.Types module as DAML-LF 1.6 (#4095)
This was accidentally set to 1.7 before for no reason. Note that this
simply meant that we did not use the stable module when compiling with
--target=1.6 since we filter out newer stable packages.

Nevertheless, I’ve also added a test that verifies that we do not
accidentally introduce dependencies on packages in newer LF versions.

changelog_begin
changelog_end
2020-01-17 18:05:39 +00:00
associahedron
df2c7c4e34 damldocs: Fix handling of constraint tuples. (#4092)
* Add damldocs golden test for constraint tuples

* Fix constraint check in Extract.hs

changelog_begin
changelog_end

* Document isConstraintType

* Refactor toText away
2020-01-17 14:57:15 +00:00
associahedron
916c70999f Improve docs around the new Template, Choice, TemplateKey constraints. (#4069)
* Try to improve the docs on DA.Internal.Template.Functions a little.

* More docs.

changelog_begin

- [DAML Standard Library] The ``Template``, ``Choice``, and
``TemplateKey`` typeclasses have been split up into many small typeclasses
to improve forward compatibility of DAML models. ``Template``,
``Choice`` and ``TemplateKey`` constraints can still be used as before.

changelog_end

* Example typo
2020-01-17 11:47:35 +00:00
associahedron
5d3040835a LF conversion of templates under new rules (WIP). (#4030)
* Patterns and test for new desugaring

* working on new template conversion

* Update ghc-lib

changelog_begin
changelog_end

* shut up hlint

* Update desugar stdlib

* update test

* remove unuseful templatebinds

* Add implicit qualified GHC.Types import

* Add missing primitives

* Remove chaff

* update comments

* Remove patterns that dont seem useful anymore

* Capture key data in template binds

* Dont make TypeRep/ToAny/FromAny classes conditional

* Remove some unnecessary TODOs

* Generate the template definition

* Remove new template desugaring test

* Fix jq query

* Rename makeDesugarDFunProjection to useSingleMethodDict

* Let TTypeRep and TAny be TUnit in primitives.

* Fix damlc visual wrt the new desugaring

* Fix visualization tests in shake test suite

* Fix damldocs

* Drop envTemplateKeyData

* Use the new ghc-lib release

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-15 18:03:15 +01:00
Moritz Kiefer
8cf4e9a9ef Allow invoking generate-src on the main dalf of a dar (#4051)
* Allow invoking generate-src on the main dalf of a dar

Given that daml build outputs a .dar file this is much more convenient
for testing against actual build outputs of damlc rather than
handcrafted .dalf files.

The behavior matches the one of `damlc inspect` in that it detects
whether it works on a .dar or a .dalf based on the file ending.

changelog_begin
changelog_end

* Update compiler/damlc/lib/DA/Cli/Damlc.hs

Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>

Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
2020-01-15 11:02:35 +00:00
Moritz Kiefer
b5508cdc5b Disable generation of Template instances (#4043)
The generation of Template instances will break horribly with
https://github.com/digital-asset/daml/pull/4030 and we don’t actually
want to use this anymore since we want to reuse old class
instances. There are some cases where we might still need or want this
in the future (see the inline comment), so for now, the code for this is still left intact and
only disabled.

changelog_begin
changelog_end
2020-01-14 17:27:42 +00:00
Moritz Kiefer
f308c433e8
Simplify building the GHC AST for data dependencies (#4042)
This PR adds a bunch of top-level helpers that take care of building
up the GHC AST.

I’ve also briefly played around with not using the GHC AST at all but
that doesn’t seem to really help simplify things so for now, I think
this is the better approach

changelog_begin
changelog_end
2020-01-14 15:50:12 +01:00
Moritz Kiefer
136c2147c0 Remove version suffix from stable packages in daml-stdlib (#4037)
Given that these packages should be stable it makes no sense to
include the version in them.

changelog_begin
changelog_end
2020-01-14 12:08:51 +00:00
Moritz Kiefer
2d0d79e889 Don’t expose primitive and magic (#4035)
Previously, we exposed those functions from Prelude which really
doesn’t make any sense given that they are intended for the internal
implementation of daml-prim and daml-stdlib.

changelog_begin
changelog_end
2020-01-14 09:42:31 +00:00
Moritz Kiefer
dacf7203f0 Respect build-options if project-root is used (#4025)
* Respect build-options if project-root is used

changelog_begin

- [DAML Compiler] The ``build-options`` field from ``daml.yaml`` is
now also respected if ``--project-root`` is used.

changelog_end

* address review comments
2020-01-13 15:18:40 +00:00
Moritz Kiefer
b42216a2fa Bump ghcide (#4013)
* Bump ghcide

changelog_begin
changelog_end

* Fix ghcide target

* fix the fix
2020-01-10 19:06:42 +00:00
nickchapman-da
006aa9b608
Type checking DAML-LF type synonyms (#3959)
* CHANGELOG_BEGIN
Type-check type synonyms.
CHANGELOG_END

* placate HLint

* comments

* Add an example that requires the check in kindOf

* check types containing syn-apps are well formed even when there is no expression of that type

* show type mismatch error after synonyms are expanded

* typeOf calls expandTypeSynonyms; track vars bound by TForall during expansion

* test interaction of syn-expansion and free-vars; add one bigger testcase

* extend bigger example with pointed typeclass, having functor as a super class

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-10 17:51:51 +00:00
Moritz Kiefer
53f25599d0 Send session ping telemetry messages (#3997)
* Send session ping telemetry messages

This PR adds telemetry information that is sent regularly (every 5
minutes) while users are active (they’ve opened, closed or changed a
file). This allows us to track how long user sessions are.

This PR does not include any additional information in the session
ping but we plan to add more in the future.

The ping messages can be found in big query by filtering for
`jsonPayload.type = "session_ping"`.

CHANGELOG_BEGIN
CHANGELOG_END

* add copyright header
2020-01-09 14:34:23 +00:00
Moritz Kiefer
ad5e05c2ce Remove Optional from internalTypes (#3981)
That list only contains types in DA.Internal.LF whereas Optional is in
DA.Internal.Prelude.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-08 14:05:58 +00:00
Moritz Kiefer
7bce23525f
Verify expected hashes of stable LF packages (#3978)
As mentioned in the comment, it doesn’t really make sense to include
the SDK version in the DALF names but I’ll address this in a separate
PR.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-08 14:34:44 +01:00
Moritz Kiefer
0ec595b680
Add regression test for unstable serializable types (#3976)
* Add regression test for unstable serializable types

This test verifies that we only have the serializable types in
daml-prim and daml-stdlib that we expect and don’t introduce new ones
by accident.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix windows

* Ignore whitespace because windows

* optional is dead
2020-01-08 12:06:07 +01:00
Moritz Kiefer
86951ea106
Remove Optional definition in LF conversion (#3973)
All references to Optional are already mapped to the builtin Optional
type in LF but we forgot to filter out the type definition.

CHANGELOG_BEGIN
CHANGELOG_END
2020-01-08 11:32:41 +01:00
Moritz Kiefer
a8b4a84b08
Move most of the remaining serializable types to stable LF packages (#3964)
* Move most of the remaining serializable types to stable LF packages

The only serializable types left in DAML stdlib after this PR are the
following:

- DA.Upgrade:MetaEquiv
- DA.Random:Minstd
- DA.Next.Set:Set
- DA.Next.Map:Map
- DA.Generics:MetaSel0
- DA.Generics:MetaData0
- DA.Generics:DecidedStrictness
- DA.Generics:SourceStrictness
- DA.Generics:SourceUnpackedness
- DA.Generics:Associativity
- DA.Generics:Infix0
- DA.Generics:Fixity
- DA.Generics:K1
- DA.Generics:Par1
- DA.Generics:U1
- DA.Internal.Prelude:Optional

Ignoring the Generics stuff which isn’t very urgent imho and the
Upgrade stuff which is probably going to change significantly anyway,
this leaves us with the weird Random module, the wrappers around
TextMap which will go away anyway and DA.Internal.Prelude:Optional
which shouldn’t exist in the first place (I’ll address that in a
separate PR).

CHANGELOG_BEGIN

- [DAML Compiler] Move more types from daml-stdlib to standalone LF
packages. The module names for the types have also changed
slightly. This only matters over the Ledger API when you specify the
module name explicitly. In DAML you should continue to use the
existing module names.

  - The types from ``DA.Semigroup` are now in a separate package under
  ``DA.Semigroup.Types``
  - The types from ``DA.Monoid` are now in a separate package under
  ``DA.Monoid.Types``
  - The types from ``DA.Time` are now in a separate package under
  ``DA.Time.Types``
  - The types from ``DA.Validation` are now in a separate package
  under ``DA.Validation.Types``
  - The types from ``DA.Logic` are now in a separate package under
  ``DA.Logic.Types``
  - The types from `DA.Date` are now in a separate package under
  `DA.Date.Types`.
  - The `Down` type from `DA.Internal.Prelude` is now in a separate
  package under `DA.Internal.Down`.

CHANGELOG_end

* Fix serializability of RelTime

* fix daml-docs

* Fix tests
2020-01-07 18:25:23 +01:00
associahedron
ec769ede31
damldocs: Add --exclude-instances option and exclude HasField instance docs from stdlib. (#3962)
* Refactor transform options

* Finish refactoring and implement --exclude-interface

* Exclude HasField instances from stdlib docs.

CHANGELOG_BEGIN

- [DAML SDK] ``daml damlc docs`` now accepts a ``--exclude-instances``
option to exclude unwanted instance docs by class name.

CHANGELOG_END

* s/excludeInstances/excludeInterfaces/g

* Review comments
2020-01-07 15:32:46 +00:00
Moritz Kiefer
42c586f8d4
Bump ghcide (#3943)
* Bump ghcide

* Fix ghcide build

* Include bugfix for Windows
2020-01-04 07:51:51 +01:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00