Commit Graph

777 Commits

Author SHA1 Message Date
associahedron
9ae41c6c84
Generate empty stub instances in data-dependencies (#4331)
Based on a comment by @cocreature in the last PR, this PR changes the
generation of typeclass instances to empty stubs, instead of adding
stub methods as well.

changelog_begin
changelog_end
2020-01-31 16:23:25 +00:00
Moritz Kiefer
ea0bf33c1c
Mark packaging tests as flaky (#4329)
Sadly my fix in https://github.com/digital-asset/daml/pull/4325 didn’t
seem to work (despite fixing the issue in my tests) so let’s mark this
as flaky until we figure it out.

changelog_begin
changelog_end
2020-01-31 15:00:25 +00:00
associahedron
132695e599
Expose instances in data-dependencies. (#4326)
* Expose instances in data-dependencies

changelog_begin
changelog_end

* Add a direct instance import test.
2020-01-31 14:20:09 +00:00
Moritz Kiefer
6a8f4aef6f
Ensure that diagnostics in buildDar are not lost (#4325)
We have seen a couple of flaky tests runs on CI in the packaging test
caused by this. It is easy to reproduce if you add a delay in
diagnosticsLogger before printing the diagnostics which makes this
failure reproducible. With this patch, the diagnostics are always
printed even with a delay.

changelog_begin
changelog_end
2020-01-31 14:00:53 +00:00
Moritz Kiefer
0f070ff34e
Remove mkOptions and defaultOptionsIO (#4323)
* Remove mkOptions and defaultOptionsIO

`mkOptions` is a terrible API and this PR burns it with lots of 🔥.

The only thing that I didn’t simply move around is the validation
logic in `mkOptions` which validates that some directories
exist. Given that I’ve never actually see this be useful and the
options to set these directories (package dbs, import paths) are
internal I don’t think we actually gain anything from this.

I’ve also killed defaultOptionsIO which was a very confusing wrapper
around `mkOptions` that in addition to calling `mkOptions` also
enabled DLint. This was only used in tests, so I’ve enabled DLint in
the tests that need this.

changelog_begin
changelog_end

* newtype version header
2020-01-31 13:39:54 +00:00
associahedron
a69528fab4
Generate module references at the same time as conversion. (#4274)
* Generate mod refs at the same time as conversion

* Small change

changelog_begin
changelog_end

* lint

* Get rid of secondM

* Switch to Writer.CPS

* doc typo
2020-01-31 12:26:31 +00:00
Tamás Kálcza
64864d9d9c
Partition function to prelude (#4318)
* Added a partition function to prelude.

* Fixed partition function documentation.

* CHANGELOG_BEGIN
- Added `partition` function to prelude.
CHANGELOG_END

* Fixed partition documentation.
2020-01-31 12:20:50 +01:00
Moritz Kiefer
65fc65ffa8
Factor out logic from mkOptions (#4319)
This is purely a refactoring in preparation of killing `mkOptions`
completely.

changelog_begin
changelog_end
2020-01-31 11:11:18 +00:00
Moritz Kiefer
104376ac65
Pass uninitialized mkOptions to initPackageDb (#4291)
`initPackageDb` assumes that `Options` has not yet been initialized
and calls `mkOptions` itself. Each call to `mkOptions` appends the LF
version to the package db dir which means that calling it twice as we
did in `execIde` results in pkg dbs of the form dir/1.7/1.7 which is
obviously not what we want.

This is just the fix, I’m sufficiently annoyed by this now, that I’ll
spend some time tomorrow to kill mkOptions completely but for now this
at least fixes the SDK on master.

The reason why we didn’t catch this in our tests is that the package
dbs are located slightly differently via Bazel runfiles as they are
located in the final release tarball which ended up not breaking this
in our LSP tests.

changelog_begin
changelog_end
2020-01-30 19:14:16 +00:00
Shayne Fletcher
a48b7b039f
Update hlint (#4276)
* Update hlint

changelog_begin
changelog_end

* Update hlint

changelog_begin
changelog_end
2020-01-30 00:06:18 +00:00
Samir Talwar
4d324f6aaf
Run scalafmt on all files and fix the errors. (#4260)
Unfortunately, downgrading `scalafmt` resulted in files that would be
reformatted to invalid code, because lines would be merged but trailing
commas would be left in.

I've manually run it everywhere and fixed these issues so people don't
have to fix them as they go.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-01-29 16:15:49 +00:00
Stefano Baghino
87b4917ccf
Run scalafmt to reduce noise on #4217 (#4245)
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-29 10:18:56 +01:00
associahedron
a8343e62c6 Refactor ModRef out as a type. (#4251)
* Refactor modrefs out as a type.

This is in preparation for moving to a monadic style.

changelog_begin
changelog_end

* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/DataDependencies.hs

Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-28 18:17:49 +00:00
associahedron
aeaf3ceab7
Export typeclasses in data-dependencies. (#4247)
* Typeclass declarations

* Add modrefs for typesyns

changelog_begin
changelog_end

* Convert constraints and expose methods using them
2020-01-28 16:44:54 +00:00
Martin Huschenbett
d568c113df
damlc: Allow all build options for damlc ide (#4213)
Currently, the DAML language server use for DAML Studio crashes when the
`build-options` section of `daml.yaml` contains flags like `--package`.
This PR fixes this issue by allowing most of the flags for `damlc build`
for `damlc ide` as well.

CHANGELOG_BEGIN

- [DAML Studio] Allow all command line flags for ``daml build`` for
  DAML Studio as well.

CHANGELOG_END
2020-01-28 17:22:02 +01:00
Andreas Herrmann
74984559e8 Test triggers in scenarios (#4233)
* Bazel test for trigger scenario

* daml-triggers: Allow testing trigger rules in scenarios

CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END

* Allow assertions on create commands

CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END

* Add convenience to construct ACS for testRule

* Add assertions for exercise and exerciseByKey

* fix assert message

* Test assertExercise(ByKey)Cmd

* unpackCommands --> flattenCommands

* Add API documentation

* Document that command ids start from "0"

* generalise command assertions to CanAbortm

* export ACSBuilder type

* Better haddocs for trigger command assertions

* explicit let

* ./fmt.sh

* Fix runfiles on Windows

* Add reference to Bazel issue

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-28 11:50:46 +00:00
Moritz Kiefer
54ace2ce06 Initialize package db in damlc test (#4240)
fixes #3436

changelog_begin

- [DAML Compiler] ``damlc test`` now initializes the packagedb automatically which
  means that it will work on projects that declare custom
  ``dependencies`` in ``daml.yaml`` without having to call ``damlc
  init`` first.

changelog_end
2020-01-28 11:22:25 +00:00
associahedron
02b531d254 Preserve class method names during LF conversion. (#4230)
* Preserve class method names during LF conversion.

This PR changes the LF conversion to pick names for dictionary
fields that will be easier to reconstruct as typeclasses
later.

For superclasses this uses "s_1", "s_2", "s_3" and so on,
and for methods this uses "m_foo", "m_bar", "m_baz" and
so on. (Is this enough C++?)

In the future we might want to distinguish between methods
that are mandatory and methods that are optional ... but
I think this should be good enough for now.

This PR also adds a test.

Fixes #4216

changelog_begin
changelog_end

* lint

* Run test in LF 1.7 as well

* Add test for superclass field.
2020-01-28 10:55:02 +00:00
Moritz Kiefer
8a67aff8de Remove position-mapping tests (#4238)
They have been migrated to `ghcide` in
https://github.com/digital-asset/ghcide/pull/388 which is a much more
sensible place for them to live given that the code is in that repo.

changelog_begin
changelog_end
2020-01-28 10:08:15 +00:00
Moritz Kiefer
3245a7b587 Fail when a data-dependency fails to compile (#4219)
* Fail when a data-dependency fails to compile

Previously, we emitted errors but just kept on going which sometimes
kinda works but is certainly not the right thing to do.

Now we just crash and abort compilation.

changelog_begin
changelog_end

* why is windows

* undo crap
2020-01-27 15:25:47 +00:00
Moritz Kiefer
40292c2b6d
Avoid including daml-stdlib and daml-prim twice (#4222)
This fixes #4114 and cleans up the situation around
`data-dependencies` and `dependencies` as described in #4218.

There is still more work to be done here mostly around ironing out all
the edge cases and producing useful error messages instead of
silentely doing the wrong thing but I’ll leave that to a separate PR.

To test this, I’ve fixed the packaging tests to no longer deduplicate
package ids (which means they would return the wrong number if
daml-prim ends up in there twice) and I addded a test where we have 3
projects:

- `lib`
- `a` which depends on `lib`
- `b` which depends on `lib` via `dependencies` and 'a' via
  `data-dependencies`.

changelog_begin
changelog_end
2020-01-27 14:56:12 +01:00
associahedron
830c2c65f5 Add missing CurrentSdk prefixes in data-dependencies (#4220)
* Expose scenarios in data-dependencies.

Also add some type signatures.

changelog_begin
changelog_end

* Add missing prefixes in data-dependencies
2020-01-27 12:57:35 +01:00
associahedron
3c93b5e6c9
Converting functions in data dependencies. (#4182)
* Converting functions in data dependencies.

changelog_begin
changelog_end

* Add NoOverloadedStrings to the set of extensions in generated packages.

* Just use the fn as its own right hand side

* Restore order of package map (not that it makes a difference)

* Adjust imports

* Weird lint but ok

* Make the test pass somehow

* Dont preprocess enums in GHC.Prim

* Preprocess enums everywhere, and add mod ref as needed.

* Revert preprocessor changes

* Dont expose old-style typeclasses

* Dont convert newstyle typeclasses temporarily either

* Add test for function importing
2020-01-27 10:05:00 +00:00
Moritz Kiefer
3496ce03a4 Convert type classes to LF type synonyms (#4023)
changelog_begin
changelog_end
2020-01-24 09:24:45 +00:00
Moritz Kiefer
93d7b1a472 Bump ghcide to fix GHC panics in LSP tests (#4176)
fixes #4152

changelog_begin
changelog_end
2020-01-24 08:40:57 +00:00
nickchapman-da
62d592ec62
support type synonyms in scala (#4101)
* Support DAML-LF type synonyms in scala.

CHANGELOG_BEGIN
CHANGELOG_END

* dont create synonymns in GenerateSimpleDalf

* extend DAML-LF parser to support type synonyms

* test: expand type synonyms correctly
2020-01-23 10:49:28 +00:00
Shayne Fletcher
e9d67ba9f6
Disambiguate consuming vs pre-consuming choices (#4164)
changelog_begin
- [DAML Compiler] Choices marked explicitly as `preconsuming` are now equivalent to a non-consuming choice that calles `archive self` at the beginning.
changelog_end

Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
2020-01-22 14:24:00 -05:00
Remy
b6e848b576
Engine: Add node and contract Discriminator
CHANGELOG_BEGIN
CHANGELOG_END
2020-01-22 18:49:30 +01:00
Moritz Kiefer
2fb107b505 Remove damlc.jar (#4157)
* Remove damlc.jar

We never advertised or published this externally and our only internal
user has moved off this months ago already.

changelog_begin
changelog_end

* Remove dependency from navigator test lib on damlc jar
2020-01-22 16:44:27 +00:00
Moritz Kiefer
57d0e32c3e
Mark lsp-tests as flaky (#4158)
This makes me a :sadpanda: but until we have figured out what is going
on, this seems better than forcing manual restarts.

changelog_begin
changelog_end
2020-01-22 16:55:44 +01:00
Shayne Fletcher
674eaffc70
Depend on ghc-lib-parser-ex (for hlint) (#4109)
changelog_begin
changelog_end

Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
2020-01-22 06:25:51 -05:00
Moritz Kiefer
19dd7ed7c4 Handle unserializable scenario result types (#4156)
* Handle unserializable scenario result types

Previously, the conversions made some attempt at guarding against this
by matching on SPap and PClosure. While it would be possible to extend
this to match on STypeRep, this doesn’t actually fix the issue since
this can be nested, e.g., you can have a record with a field that is
an SPAP.

This PR changes this to simply catch any errors thrown from
`toValue`. While this feels a bit ugly, I think it’s a reasonable fix
for now.

changelog_begin

- [DAML Studio] Scenarios with unserializable result types no longer
  crash the scenario service.

changelog_end

* Only run test on DAML-LF >= 1.7
2020-01-22 11:15:46 +00:00
Moritz Kiefer
8d7c021001 Remove intermediate PackageImport type (#4142)
* Remove intermediate PackageImport type

We can just reuse GHC’s PackageFlag directly which simplifies things a
bit.

changelog_begin
changelog_end

* windows is bad

* meh

* foobar

* why
2020-01-21 20:27:26 +00:00
Moritz Kiefer
359e5c350c
Fix enums in data-dependencies (#4143)
To differentiate between "data X = X" which is translated to a DAML-LF
enum and "data X = X {}" which is translated to a DAML-LF record, we
add a datatype context with GHC.Types.DamlEnum. For
`data-dependencies` this needs to point to Currentsdk.GHC.Types.

changelog_begin
changelog_end
2020-01-21 16:11:31 +01:00
Moritz Kiefer
ca3da8ac7c
Use GHC’s syntax for the --package flag (#4136)
* Use GHC’s syntax for the --package flag

The previous syntax was just based on the Read instance of a 3-tuple
which was pretty much unusable. This changes it to GHC’s much nicer
syntax.

I deliberately did not add a changelog entry for this, since this
flag isn’t something that we have documented at all and it is
currently only useful in combination with data-dependencies.

fixes #4126

changelog_begin
changelog_end

* Fix hlint

* get it to compile 😇

* shut up hlint
2020-01-21 13:53:54 +01:00
Moritz Kiefer
dc3953e52f
Remove the key field from pkg.conf files (#4127)
* Remove the key field from pkg.conf files

We never use this and it only a legacy field that is deprecated in
favor of id in GHC.

There is also a minor change in the docs around ExposePackage which
should at most affect error messages.

changelog_begin
changelog_end

* Better error messages
2020-01-21 13:30:38 +01:00
Moritz Kiefer
7525e73134 Bump timeout of packaging tests (#4137)
The packaging tests have grown a fair bit recently and seem to hit
timeouts sometimes. We might want to split it up into multiple tests
suites but for now this should help.

changelog_begin
changelog_end
2020-01-21 11:18:09 +00:00
Moritz Kiefer
7f4061d55c
Remove code for generation of Template and Choice instances (#4131)
The current code has gotten out of sync with the changes to template
desugaring and is therefore simply broken. In addition to that, we
don’t need this once we can use typeclass instances in combination
with data dependencies.

Initially, we just disabled this instead of removing it since we
thought it might still be useful for older packages before typeclasses
are desugared to type synonyms. But we already have one way of
handling those as evidenced in the tests for the DAVL and once we can
call functions from older packages (which works regardless of whether
they have been built before or after the changes to how typeclasses
are desugared), we have another one so I don’t think it makes any
sense to keep this code around.

changelog_begin
changelog_end
2020-01-21 11:57:08 +01:00
associahedron
d9220c6819 damldocs: Refactoring DA.Daml.Doc.Extract (#4112)
* Refactor damldocs

* Refactor damldocs.

changelog_begin
changelog_end

* More refactoring

* Update copyright headers.

* Add export list for Templates

* Add export list to .Util

* added a little ^
2020-01-20 16:17:25 +00:00
Moritz Kiefer
4804fca5d5 Test data-dependencies for more LF version combinations (#4113)
* Test data-dependencies for more LF version combinations

This PR extends the cross-LF version tests to run for all
combinations where depLfVer >= targetLfVer.

changelog_begin
changelog_end

* Fix comments and add TODO
2020-01-20 14:59:43 +00:00
Moritz Kiefer
67f028b0e8
Support cross-SDK data-dependencies against DAVL (#4107)
This PR fixes a minor issue where we were including more references
than we need which causes problems on older SDKs where GHC.Prim is not
a stable module since GHC.Prim cannot be imported (it’s builtin GHC
magic).

More importantly, this PR adds a test for cross-SDK data-dependencies
against the DAVL DAR which is built with SDK 0.13.32 before we started
making all the changes. The test also includes a slightly hacky but
mostly reasonable way of actually using the templates instead of just
the data types from the DAVL DAR which and a scenario to test this.

changelog_begin
changelog_end
2020-01-20 14:54:02 +01:00
associahedron
4a25c8c166
damldocs: Add a --drop-orphan-instances flag (#4100)
* damldocs: Add a --drop-orphan-instances flag

CHANGELOG_BEGIN

- [DAML SDK] Added a ``--drop-orphan-instances`` flag in ``daml damlc
docs``.

CHANGELOG_END

* Update copyright headers

* Document MOVE annotation better and improve orphan instance filter
2020-01-20 10:15:20 +00:00
Moritz Kiefer
9ece831966
Make DAR generation deterministic (#4104)
This fixes the ZIP modification times in all DARs to a specific
value (1980-01-01) whereas they used the current time before. This
both gives us the nice property that not only our DALF builds but also
DAR builds should be deterministic (and there is a test for this). I
have a suspicion that this could help significantly with build times
and avoid rerunning half of the Scala tests on a change to damlc that
should not change the DALFs.

changelog_begin

- [DAML Compiler] The modification times in a DAR are now fixed to a
  given value which makes the output of ``daml build`` deterministic
  in single-threaded mode (which is the default).

changelog_end
2020-01-20 11:00:10 +01:00
Moritz Kiefer
fa8a92f772
Make data-dependencies work with packages produced by damlc (#4099)
This PR fixes a whole bunch of bugs when using data-dependencies on a
package produced by damlc rather than the handcrafted simple-dalf.

The end result is that we are able to build a package with
`--target=1.6` and use it as a data-dependency from a package with
`-target=1.7`. While the test that I’ve added for this is not
cross-SDK it is arguably even trickier since we cannot rely on the
unit id of daml-stdlib changing the version in the unit id.

The changes all fall into one of the following two categories:

- Special handling for daml-prim and daml-stdlib to avoid collisions
  with the one from the current SDK.
- Special treatment of stable packages to avoid generating new code
  for those.
- Remove a couple of filters for daml-prim since we do need to include
  daml-prim from a data-dependency if it is a different version.

This is definitely not perfect and there are a bunch of rather hacky
parts in here but it seems like a good intermediate step (and it has a
test :))

changelog_begin
changelog_end
2020-01-17 20:09:30 +01:00
Moritz Kiefer
f77e326e0b Generate stable DA.Time.Types module as DAML-LF 1.6 (#4095)
This was accidentally set to 1.7 before for no reason. Note that this
simply meant that we did not use the stable module when compiling with
--target=1.6 since we filter out newer stable packages.

Nevertheless, I’ve also added a test that verifies that we do not
accidentally introduce dependencies on packages in newer LF versions.

changelog_begin
changelog_end
2020-01-17 18:05:39 +00:00
associahedron
df2c7c4e34 damldocs: Fix handling of constraint tuples. (#4092)
* Add damldocs golden test for constraint tuples

* Fix constraint check in Extract.hs

changelog_begin
changelog_end

* Document isConstraintType

* Refactor toText away
2020-01-17 14:57:15 +00:00
associahedron
916c70999f Improve docs around the new Template, Choice, TemplateKey constraints. (#4069)
* Try to improve the docs on DA.Internal.Template.Functions a little.

* More docs.

changelog_begin

- [DAML Standard Library] The ``Template``, ``Choice``, and
``TemplateKey`` typeclasses have been split up into many small typeclasses
to improve forward compatibility of DAML models. ``Template``,
``Choice`` and ``TemplateKey`` constraints can still be used as before.

changelog_end

* Example typo
2020-01-17 11:47:35 +00:00
Moritz Kiefer
74d12ce39a Bump ghcide to include bugfix (#4074)
* Bump ghcide to include bugfix

This includes a fix for a bug in completions which we introduced in
the latest ghcide bump which in turn broke the DAML IDE completely.

changelog_begin

- [DAML Studio] Fix a bug in completions that caused DAML Studio to
stop responding after the first completion was requested.

changelog_end

* Use ghcide from master branch and simplify test
2020-01-16 18:38:58 +00:00
Robin Krom
0a26591849
upgrading to newest nodejs_rules (#4057)
* upgrading to newest nodejs_rules

CHANGELOG_BEGIN
CHANGELOG_END

* addressing andreas comments
2020-01-16 15:55:32 +01:00
associahedron
5d3040835a LF conversion of templates under new rules (WIP). (#4030)
* Patterns and test for new desugaring

* working on new template conversion

* Update ghc-lib

changelog_begin
changelog_end

* shut up hlint

* Update desugar stdlib

* update test

* remove unuseful templatebinds

* Add implicit qualified GHC.Types import

* Add missing primitives

* Remove chaff

* update comments

* Remove patterns that dont seem useful anymore

* Capture key data in template binds

* Dont make TypeRep/ToAny/FromAny classes conditional

* Remove some unnecessary TODOs

* Generate the template definition

* Remove new template desugaring test

* Fix jq query

* Rename makeDesugarDFunProjection to useSingleMethodDict

* Let TTypeRep and TAny be TUnit in primitives.

* Fix damlc visual wrt the new desugaring

* Fix visualization tests in shake test suite

* Fix damldocs

* Drop envTemplateKeyData

* Use the new ghc-lib release

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-01-15 18:03:15 +01:00