This adds the NodeInfo trait to compute the informees of a node and the
required authorizers. This has been separated into its own trait as this
computation is also required (externally) for serialized transaction nodes
in a context where the full deserialization of the node would be prohibitively
expensive.
While doing this change and cleaning up transaction authorization it was
discovered that top-level fetches (which can only be created from scenarios)
did not include the submitter as a witnessing party.
CHANGELOG_BEGIN
[DAML SDK] Fix computation of witnesses of top-level fetch nodes in scenario results ("known to").
CHANGELOG_END
Previously, we did not send any message when users simply clicked away
the telemetry popup. Now we send a special message similar to the
opt-out message which only includes the machine id.
I’ve also changed the opt-out message to include the machine id.
changelog_begin
changelog_end
* Use a safer safeToReexport
This is much safer than the approximation from last time. The only downside is
introducing a dependency between data dependencies and our type checker,
but that seems safer than having two versions of `expandTypeSynonyms`
floating around (and perhaps this dependency is something we would end
up adding anyway).
changelog_begin
changelog_end
* Add own package to extPackages.
* Use mkTForalls
* simplify mkTForalls
* Re-export typeclasses in data-dependencies...
... whenever possible.
The `safeToReexport` over-approximation is simple but troubling, so
I'd like to get it fixed soon.
The inputs to generateSrcFromLfPkg (and the genInstances one) were
getting quite hairy, so I decided to pass a record instead.
changelog_begin
changelog_end
* Add a test
* Move generateSrcPkgFromLf to its original place
* Better docs and make envQualify more clear
* compute dependency ids in generate-src command
The packaging tests are already one of our slowest test suites and
damlc test takes quite a while since it has to spin up the scenario
service.
We already have tests for damlc tests so this PR moves the tests from
the packaging test suite that are specific to `damlc test` to those
tests which should balance things a bit better.
changelog_begin
changelog_end
Based on a comment by @cocreature in the last PR, this PR changes the
generation of typeclass instances to empty stubs, instead of adding
stub methods as well.
changelog_begin
changelog_end
Sadly my fix in https://github.com/digital-asset/daml/pull/4325 didn’t
seem to work (despite fixing the issue in my tests) so let’s mark this
as flaky until we figure it out.
changelog_begin
changelog_end
We have seen a couple of flaky tests runs on CI in the packaging test
caused by this. It is easy to reproduce if you add a delay in
diagnosticsLogger before printing the diagnostics which makes this
failure reproducible. With this patch, the diagnostics are always
printed even with a delay.
changelog_begin
changelog_end
* Remove mkOptions and defaultOptionsIO
`mkOptions` is a terrible API and this PR burns it with lots of 🔥.
The only thing that I didn’t simply move around is the validation
logic in `mkOptions` which validates that some directories
exist. Given that I’ve never actually see this be useful and the
options to set these directories (package dbs, import paths) are
internal I don’t think we actually gain anything from this.
I’ve also killed defaultOptionsIO which was a very confusing wrapper
around `mkOptions` that in addition to calling `mkOptions` also
enabled DLint. This was only used in tests, so I’ve enabled DLint in
the tests that need this.
changelog_begin
changelog_end
* newtype version header
* Generate mod refs at the same time as conversion
* Small change
changelog_begin
changelog_end
* lint
* Get rid of secondM
* Switch to Writer.CPS
* doc typo
* Added a partition function to prelude.
* Fixed partition function documentation.
* CHANGELOG_BEGIN
- Added `partition` function to prelude.
CHANGELOG_END
* Fixed partition documentation.
`initPackageDb` assumes that `Options` has not yet been initialized
and calls `mkOptions` itself. Each call to `mkOptions` appends the LF
version to the package db dir which means that calling it twice as we
did in `execIde` results in pkg dbs of the form dir/1.7/1.7 which is
obviously not what we want.
This is just the fix, I’m sufficiently annoyed by this now, that I’ll
spend some time tomorrow to kill mkOptions completely but for now this
at least fixes the SDK on master.
The reason why we didn’t catch this in our tests is that the package
dbs are located slightly differently via Bazel runfiles as they are
located in the final release tarball which ended up not breaking this
in our LSP tests.
changelog_begin
changelog_end
Unfortunately, downgrading `scalafmt` resulted in files that would be
reformatted to invalid code, because lines would be merged but trailing
commas would be left in.
I've manually run it everywhere and fixed these issues so people don't
have to fix them as they go.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Refactor modrefs out as a type.
This is in preparation for moving to a monadic style.
changelog_begin
changelog_end
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/DataDependencies.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Currently, the DAML language server use for DAML Studio crashes when the
`build-options` section of `daml.yaml` contains flags like `--package`.
This PR fixes this issue by allowing most of the flags for `damlc build`
for `damlc ide` as well.
CHANGELOG_BEGIN
- [DAML Studio] Allow all command line flags for ``daml build`` for
DAML Studio as well.
CHANGELOG_END
* Bazel test for trigger scenario
* daml-triggers: Allow testing trigger rules in scenarios
CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END
* Allow assertions on create commands
CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END
* Add convenience to construct ACS for testRule
* Add assertions for exercise and exerciseByKey
* fix assert message
* Test assertExercise(ByKey)Cmd
* unpackCommands --> flattenCommands
* Add API documentation
* Document that command ids start from "0"
* generalise command assertions to CanAbortm
* export ACSBuilder type
* Better haddocs for trigger command assertions
* explicit let
* ./fmt.sh
* Fix runfiles on Windows
* Add reference to Bazel issue
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
fixes#3436
changelog_begin
- [DAML Compiler] ``damlc test`` now initializes the packagedb automatically which
means that it will work on projects that declare custom
``dependencies`` in ``daml.yaml`` without having to call ``damlc
init`` first.
changelog_end
* Preserve class method names during LF conversion.
This PR changes the LF conversion to pick names for dictionary
fields that will be easier to reconstruct as typeclasses
later.
For superclasses this uses "s_1", "s_2", "s_3" and so on,
and for methods this uses "m_foo", "m_bar", "m_baz" and
so on. (Is this enough C++?)
In the future we might want to distinguish between methods
that are mandatory and methods that are optional ... but
I think this should be good enough for now.
This PR also adds a test.
Fixes#4216
changelog_begin
changelog_end
* lint
* Run test in LF 1.7 as well
* Add test for superclass field.
They have been migrated to `ghcide` in
https://github.com/digital-asset/ghcide/pull/388 which is a much more
sensible place for them to live given that the code is in that repo.
changelog_begin
changelog_end
* Fail when a data-dependency fails to compile
Previously, we emitted errors but just kept on going which sometimes
kinda works but is certainly not the right thing to do.
Now we just crash and abort compilation.
changelog_begin
changelog_end
* why is windows
* undo crap
This fixes#4114 and cleans up the situation around
`data-dependencies` and `dependencies` as described in #4218.
There is still more work to be done here mostly around ironing out all
the edge cases and producing useful error messages instead of
silentely doing the wrong thing but I’ll leave that to a separate PR.
To test this, I’ve fixed the packaging tests to no longer deduplicate
package ids (which means they would return the wrong number if
daml-prim ends up in there twice) and I addded a test where we have 3
projects:
- `lib`
- `a` which depends on `lib`
- `b` which depends on `lib` via `dependencies` and 'a' via
`data-dependencies`.
changelog_begin
changelog_end
* Converting functions in data dependencies.
changelog_begin
changelog_end
* Add NoOverloadedStrings to the set of extensions in generated packages.
* Just use the fn as its own right hand side
* Restore order of package map (not that it makes a difference)
* Adjust imports
* Weird lint but ok
* Make the test pass somehow
* Dont preprocess enums in GHC.Prim
* Preprocess enums everywhere, and add mod ref as needed.
* Revert preprocessor changes
* Dont expose old-style typeclasses
* Dont convert newstyle typeclasses temporarily either
* Add test for function importing
* Support DAML-LF type synonyms in scala.
CHANGELOG_BEGIN
CHANGELOG_END
* dont create synonymns in GenerateSimpleDalf
* extend DAML-LF parser to support type synonyms
* test: expand type synonyms correctly
changelog_begin
- [DAML Compiler] Choices marked explicitly as `preconsuming` are now equivalent to a non-consuming choice that calles `archive self` at the beginning.
changelog_end
Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
* Remove damlc.jar
We never advertised or published this externally and our only internal
user has moved off this months ago already.
changelog_begin
changelog_end
* Remove dependency from navigator test lib on damlc jar
This makes me a :sadpanda: but until we have figured out what is going
on, this seems better than forcing manual restarts.
changelog_begin
changelog_end
* Handle unserializable scenario result types
Previously, the conversions made some attempt at guarding against this
by matching on SPap and PClosure. While it would be possible to extend
this to match on STypeRep, this doesn’t actually fix the issue since
this can be nested, e.g., you can have a record with a field that is
an SPAP.
This PR changes this to simply catch any errors thrown from
`toValue`. While this feels a bit ugly, I think it’s a reasonable fix
for now.
changelog_begin
- [DAML Studio] Scenarios with unserializable result types no longer
crash the scenario service.
changelog_end
* Only run test on DAML-LF >= 1.7
* Remove intermediate PackageImport type
We can just reuse GHC’s PackageFlag directly which simplifies things a
bit.
changelog_begin
changelog_end
* windows is bad
* meh
* foobar
* why
To differentiate between "data X = X" which is translated to a DAML-LF
enum and "data X = X {}" which is translated to a DAML-LF record, we
add a datatype context with GHC.Types.DamlEnum. For
`data-dependencies` this needs to point to Currentsdk.GHC.Types.
changelog_begin
changelog_end
* Use GHC’s syntax for the --package flag
The previous syntax was just based on the Read instance of a 3-tuple
which was pretty much unusable. This changes it to GHC’s much nicer
syntax.
I deliberately did not add a changelog entry for this, since this
flag isn’t something that we have documented at all and it is
currently only useful in combination with data-dependencies.
fixes#4126
changelog_begin
changelog_end
* Fix hlint
* get it to compile 😇
* shut up hlint
* Remove the key field from pkg.conf files
We never use this and it only a legacy field that is deprecated in
favor of id in GHC.
There is also a minor change in the docs around ExposePackage which
should at most affect error messages.
changelog_begin
changelog_end
* Better error messages
The packaging tests have grown a fair bit recently and seem to hit
timeouts sometimes. We might want to split it up into multiple tests
suites but for now this should help.
changelog_begin
changelog_end
The current code has gotten out of sync with the changes to template
desugaring and is therefore simply broken. In addition to that, we
don’t need this once we can use typeclass instances in combination
with data dependencies.
Initially, we just disabled this instead of removing it since we
thought it might still be useful for older packages before typeclasses
are desugared to type synonyms. But we already have one way of
handling those as evidenced in the tests for the DAVL and once we can
call functions from older packages (which works regardless of whether
they have been built before or after the changes to how typeclasses
are desugared), we have another one so I don’t think it makes any
sense to keep this code around.
changelog_begin
changelog_end
* Refactor damldocs
* Refactor damldocs.
changelog_begin
changelog_end
* More refactoring
* Update copyright headers.
* Add export list for Templates
* Add export list to .Util
* added a little ^
* Test data-dependencies for more LF version combinations
This PR extends the cross-LF version tests to run for all
combinations where depLfVer >= targetLfVer.
changelog_begin
changelog_end
* Fix comments and add TODO
This PR fixes a minor issue where we were including more references
than we need which causes problems on older SDKs where GHC.Prim is not
a stable module since GHC.Prim cannot be imported (it’s builtin GHC
magic).
More importantly, this PR adds a test for cross-SDK data-dependencies
against the DAVL DAR which is built with SDK 0.13.32 before we started
making all the changes. The test also includes a slightly hacky but
mostly reasonable way of actually using the templates instead of just
the data types from the DAVL DAR which and a scenario to test this.
changelog_begin
changelog_end
This fixes the ZIP modification times in all DARs to a specific
value (1980-01-01) whereas they used the current time before. This
both gives us the nice property that not only our DALF builds but also
DAR builds should be deterministic (and there is a test for this). I
have a suspicion that this could help significantly with build times
and avoid rerunning half of the Scala tests on a change to damlc that
should not change the DALFs.
changelog_begin
- [DAML Compiler] The modification times in a DAR are now fixed to a
given value which makes the output of ``daml build`` deterministic
in single-threaded mode (which is the default).
changelog_end
This PR fixes a whole bunch of bugs when using data-dependencies on a
package produced by damlc rather than the handcrafted simple-dalf.
The end result is that we are able to build a package with
`--target=1.6` and use it as a data-dependency from a package with
`-target=1.7`. While the test that I’ve added for this is not
cross-SDK it is arguably even trickier since we cannot rely on the
unit id of daml-stdlib changing the version in the unit id.
The changes all fall into one of the following two categories:
- Special handling for daml-prim and daml-stdlib to avoid collisions
with the one from the current SDK.
- Special treatment of stable packages to avoid generating new code
for those.
- Remove a couple of filters for daml-prim since we do need to include
daml-prim from a data-dependency if it is a different version.
This is definitely not perfect and there are a bunch of rather hacky
parts in here but it seems like a good intermediate step (and it has a
test :))
changelog_begin
changelog_end
This was accidentally set to 1.7 before for no reason. Note that this
simply meant that we did not use the stable module when compiling with
--target=1.6 since we filter out newer stable packages.
Nevertheless, I’ve also added a test that verifies that we do not
accidentally introduce dependencies on packages in newer LF versions.
changelog_begin
changelog_end
* Try to improve the docs on DA.Internal.Template.Functions a little.
* More docs.
changelog_begin
- [DAML Standard Library] The ``Template``, ``Choice``, and
``TemplateKey`` typeclasses have been split up into many small typeclasses
to improve forward compatibility of DAML models. ``Template``,
``Choice`` and ``TemplateKey`` constraints can still be used as before.
changelog_end
* Example typo
* Bump ghcide to include bugfix
This includes a fix for a bug in completions which we introduced in
the latest ghcide bump which in turn broke the DAML IDE completely.
changelog_begin
- [DAML Studio] Fix a bug in completions that caused DAML Studio to
stop responding after the first completion was requested.
changelog_end
* Use ghcide from master branch and simplify test
* Patterns and test for new desugaring
* working on new template conversion
* Update ghc-lib
changelog_begin
changelog_end
* shut up hlint
* Update desugar stdlib
* update test
* remove unuseful templatebinds
* Add implicit qualified GHC.Types import
* Add missing primitives
* Remove chaff
* update comments
* Remove patterns that dont seem useful anymore
* Capture key data in template binds
* Dont make TypeRep/ToAny/FromAny classes conditional
* Remove some unnecessary TODOs
* Generate the template definition
* Remove new template desugaring test
* Fix jq query
* Rename makeDesugarDFunProjection to useSingleMethodDict
* Let TTypeRep and TAny be TUnit in primitives.
* Fix damlc visual wrt the new desugaring
* Fix visualization tests in shake test suite
* Fix damldocs
* Drop envTemplateKeyData
* Use the new ghc-lib release
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Allow invoking generate-src on the main dalf of a dar
Given that daml build outputs a .dar file this is much more convenient
for testing against actual build outputs of damlc rather than
handcrafted .dalf files.
The behavior matches the one of `damlc inspect` in that it detects
whether it works on a .dar or a .dalf based on the file ending.
changelog_begin
changelog_end
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
The generation of Template instances will break horribly with
https://github.com/digital-asset/daml/pull/4030 and we don’t actually
want to use this anymore since we want to reuse old class
instances. There are some cases where we might still need or want this
in the future (see the inline comment), so for now, the code for this is still left intact and
only disabled.
changelog_begin
changelog_end
This PR adds a bunch of top-level helpers that take care of building
up the GHC AST.
I’ve also briefly played around with not using the GHC AST at all but
that doesn’t seem to really help simplify things so for now, I think
this is the better approach
changelog_begin
changelog_end
Previously, we exposed those functions from Prelude which really
doesn’t make any sense given that they are intended for the internal
implementation of daml-prim and daml-stdlib.
changelog_begin
changelog_end
* Respect build-options if project-root is used
changelog_begin
- [DAML Compiler] The ``build-options`` field from ``daml.yaml`` is
now also respected if ``--project-root`` is used.
changelog_end
* address review comments
* CHANGELOG_BEGIN
Type-check type synonyms.
CHANGELOG_END
* placate HLint
* comments
* Add an example that requires the check in kindOf
* check types containing syn-apps are well formed even when there is no expression of that type
* show type mismatch error after synonyms are expanded
* typeOf calls expandTypeSynonyms; track vars bound by TForall during expansion
* test interaction of syn-expansion and free-vars; add one bigger testcase
* extend bigger example with pointed typeclass, having functor as a super class
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Send session ping telemetry messages
This PR adds telemetry information that is sent regularly (every 5
minutes) while users are active (they’ve opened, closed or changed a
file). This allows us to track how long user sessions are.
This PR does not include any additional information in the session
ping but we plan to add more in the future.
The ping messages can be found in big query by filtering for
`jsonPayload.type = "session_ping"`.
CHANGELOG_BEGIN
CHANGELOG_END
* add copyright header
As mentioned in the comment, it doesn’t really make sense to include
the SDK version in the DALF names but I’ll address this in a separate
PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Add regression test for unstable serializable types
This test verifies that we only have the serializable types in
daml-prim and daml-stdlib that we expect and don’t introduce new ones
by accident.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix windows
* Ignore whitespace because windows
* optional is dead
All references to Optional are already mapped to the builtin Optional
type in LF but we forgot to filter out the type definition.
CHANGELOG_BEGIN
CHANGELOG_END
* Move most of the remaining serializable types to stable LF packages
The only serializable types left in DAML stdlib after this PR are the
following:
- DA.Upgrade:MetaEquiv
- DA.Random:Minstd
- DA.Next.Set:Set
- DA.Next.Map:Map
- DA.Generics:MetaSel0
- DA.Generics:MetaData0
- DA.Generics:DecidedStrictness
- DA.Generics:SourceStrictness
- DA.Generics:SourceUnpackedness
- DA.Generics:Associativity
- DA.Generics:Infix0
- DA.Generics:Fixity
- DA.Generics:K1
- DA.Generics:Par1
- DA.Generics:U1
- DA.Internal.Prelude:Optional
Ignoring the Generics stuff which isn’t very urgent imho and the
Upgrade stuff which is probably going to change significantly anyway,
this leaves us with the weird Random module, the wrappers around
TextMap which will go away anyway and DA.Internal.Prelude:Optional
which shouldn’t exist in the first place (I’ll address that in a
separate PR).
CHANGELOG_BEGIN
- [DAML Compiler] Move more types from daml-stdlib to standalone LF
packages. The module names for the types have also changed
slightly. This only matters over the Ledger API when you specify the
module name explicitly. In DAML you should continue to use the
existing module names.
- The types from ``DA.Semigroup` are now in a separate package under
``DA.Semigroup.Types``
- The types from ``DA.Monoid` are now in a separate package under
``DA.Monoid.Types``
- The types from ``DA.Time` are now in a separate package under
``DA.Time.Types``
- The types from ``DA.Validation` are now in a separate package
under ``DA.Validation.Types``
- The types from ``DA.Logic` are now in a separate package under
``DA.Logic.Types``
- The types from `DA.Date` are now in a separate package under
`DA.Date.Types`.
- The `Down` type from `DA.Internal.Prelude` is now in a separate
package under `DA.Internal.Down`.
CHANGELOG_end
* Fix serializability of RelTime
* fix daml-docs
* Fix tests
I managed to break our Windows CI twice already due to this so while
it makes me sad, I think hardcoding those files is the more sensible
option for now.
* Move Any wrappers and Archive to stable packages
There are no actual API or functionality changes in this PR but the
logic for locating the stable packages has slightly changed since the
Any wrappers package only makes sense for LF 1.7. To address this, we
simply filter out stable packages for newer LF versions since it
doesn’t make sense to depend on those anyway.
CHANGELOG_BEGIN
- [DAML Compiler] Move ``Archive`` type to a separate DALF.
CHANGELOG_END
* More comments
* Fix java codegen tests
* fix more tests
* Force cache reset on Windows
* Revert "Force cache reset on Windows"
This reverts commit 9f2b7d70b2.
* Fix roundCommercial
fixes#3884
Previously, we relieid on being able to convert to an Int, which
doesn’t work since there are Numeric values that cannot be represented
as an Int. Now, we avoid any conversions to Int which fixes this
issue.
We probably just want to have a primitive for this at some point but
given that this will only be available in newer LF versions, I think
it’s worth working around this for now.
* Move tests to a separate file to exclude LF 1.6
* Update compiler/damlc/daml-stdlib-src/DA/Internal/Prelude.daml
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Apply suggestions from code review
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update comments
* Update compiler/damlc/daml-stdlib-src/DA/Internal/Prelude.daml
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Move stable types from DA.Internal.Template to a separate module
This is in preparation for splitting them into a separate LF package
with a stable package id.
* Fix visualization
* Fix data dependencies
Originally, this made sense as a flag in Options since we read it
during typechecking to write out interfaces on the fly. But now, we
write this is only used in execCompile so having this be a global
option that is ignored anywhere but in `compile` is rather confusing.
This documents some of the edge cases in the packaging code that I’ve
had trouble understanding so that the next person hopefully has an
easier time with this.
There is also some minor cleanup in this PR.
* Add Eq instances for AnyTemplate, AnyChoice and AnyContractKey
CHANGELOG_BEGIN
- [DAML Standard Library] Add ``Eq`` instances for ``AnyTemplate``, ``AnyChoice`` and ``AnyContractKey``.
CHANGELOG_END
* Add DAML_ANY_TYPE to the CPP guard
* Fix package names in depends field in pkg configs
Previously, we derived this based on the DAR name which breaks if you
use -o with rather confusing error messages. Now, we read it from the
`Name` field in the manifest that we added in
https://github.com/digital-asset/daml/pull/3805.
CHANGELOG_BEGIN
- [DAML Compiler] Fix an issue where transitive package dependencies
resulted in packages not being found, if the DAR name was changed with
`-o`.
CHANGELOG_END
* Fix package dependencies
Currently, we generate the `depends` field in the ghc-pkg config file
based on the file name of the DARs in the `dependencies` field in
`daml.yaml`. This falls apart when you use the -o option to `daml
build` since `ghc-pkg` will complain about missing packages.
This PR adds the name to the manifest so that we can generate the
`depends` field based on that but I’ll leave that change for a
separate PR.
* Update hlint
* Redefine base package to make more sense for daml
Also, elminate lens and attoparsesec packages and groups.
* Don't change the test; ignore the new lints
* More conversion haskell rules to daml
* Move all datatypes out of daml-prim
This moves the remaining two modules DA.Types and GHC.Tuple to
separate LF packages with stable identifiers.
The only data types remaining are the ones for typeclasses which will
disappear once we move this to type synonyms.
CHANGELOG_BEGIN
- [DAML Compiler] The modules DA.Types and GHC.Tuple from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Fix codegen tests
* Fix DarReader test
* Fix kvutils tests
* Fix jdbcdao tests
* Fix hs ledger bindings tests
* Full package name collision check
* Handle type synonyms appropriately
* Better comment
* Make isAscendant case-insensitive
* Document isAscendant and explain case-insensitivity
* Add a package-wide name collision test.
* Refactor packaging logic
This is a first step towards cleaning up the packaging logic and
adding some comments to make it clearer what is going on. There are no
functional changes in this PR.
There is more stuff here that we can and should cleanup but I will
leave that for separate PRs.
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Document topological sorting
* Undo requiredE change
Sadly, I haven’t managed to come up with an isolated test case for
this but I’ve tested this on a large internal codebase and it fixes an
issue with damlc build --incremental=yes.
* language: suffix all dalfs dependencies in a dar with the pkgid.
This makes sure that dalf dependencies are not accidentally overwritten
when two packages with equally named dalfs are imported.
* factor out parseUnitId
This is a first step towards making sure that the package ids for
types defined in daml-prim and daml-stdlib don’t change. This PR
mostly adds all the necessary infrastructure for that and moves
GHC.Types and GHC.Prim to make sure it works.
Until data-dependencies are really solid and we have verified that we
no longer have performance issues with an increasing number of Haskell
packages, we still include the source files in daml-prim and then just
rewrite the references.
We will also need to add tests that these packages really have stable
ids but I’ll leave that for separate PRs since this doesn’t make that
much sense anyway until all of the types have moved to stable
packages.
CHANGELOG_BEGIN
- [DAML Compiler] The modules GHC.Prim and GHC.Types from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Track used packages during whole of engine submit
- Introduce MutableCompiledPackages interface
- Add TrackingCompiledPackages that tracks fetched packages
- Make used packages in transaction optional to distinguish between
missing dependencies and empty set of dependencies.
* Reimplement package dependency tracking
- Compute direct dependencies of a package during decoding
- Compute transitive dependencies of a package when adding a package
to engine.
- Annotate the resulting transaction with package dependencies
in Engine.submit.
* Create Ast.Package with proper direct deps in scenario service
While we don't have use for direct dependencies of a package in
scenario service (only Engine.submit needs it), it's better to be
accurate.
This of course overapproximates the direct dependencies.
* Compile a each new package once in ConcurrentCompiledPackages
I’ve gone through references to generic templates in our codebase to
see what else we can drop. In this case, we are still using
coerceContractId so I’ve updated the comment to explain the current
usecase.
We needed this for our generic template hacks. Since they are gone now,
I'm very much in favour of removing this extension since "undecidable"
always sounds a bit scary, even if it is only at compile time.
CHANGELOG_BEGIN
- [DAML Compiler] Don't make `UndecidableSuperClasses` a default language
extension for DAML anymore. If you really need this feature for a module,
you can reenable it using a `LANGUAGE` pragma at the top.
CHANGELOG_END
* Introduce a simpler template desugaring without support for generic templates
This adapts the LF conversion to the new template desugaring
introduced in our GHC fork. The guiding principle is that we use the
typeclasses directly to avoid generating, typechecking and converting
redundant code caused by indirections. I updated the template
desugaring documentation so that is probably a good starting point for
reviewing this.
* Address review comments
* Fix daml doc tests
* Fix data dependency tests
* Switch to new ghc-lib release
Display the actual number the user tried to use instead of 1.2345.
The logic to display the number is not perfect but definitely better than
we have now. If somebody knows a _simple_ way to do this better, please
tell me about it.
Until now, we broke long party names in the table view in the scenario at
spaces. This looked quite ugly.
Now, we don't break them at spaces. At the same time we also moved the CSS
into a file rather than having it in the code to make changes like this one
easier in the future.
This fixes#3642.
This reenables the migration checks. The migration command now only
generates stumbs for conversions instead of using the formerly (generic)
upgrade template.
* Successfully parse rss xml and find latest blog post object
* Show notification with link to blog (on every ide startup...)
* Only show blog post if not seen before
* Formatting
CHANGELOG_BEGIN
- [SDK] Notify IDE users about new daml-driven blog posts
CHANGELOG_END
* Add setting for user to opt out of notifications
* Use await rather than then
* Catch and swallow exceptions
* Another await and minor cleanup
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.
CHANGELOG_BEGIN
- [DAML Compiler] Make the experimental feature "generic templates"
unavailable. The current implementation is at odds with other, more important
language features still under development.
CHANGELOG_END
* daml-lf: rename Map to TextMap in archive proto
+ in Scala/haskell AST
* a bit more renamming
* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Serializability.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* fix test
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Set Sdk-Version in DAR Manifest to compiler SDK version.
CHANGELOG_BEGIN
- [DAML Compiler] Bugfix: The Sdk-Version field in a DAR manifest file
now matches the SDK version of the compiler, not the sdk-version field
from daml.yaml. These are usually the same, but they could be different
if you set the DAML_SDK_VERSION environment variable before running
``daml init`` or ``daml build``.
CHANGELOG_END
* Fallback to daml.yaml if env var not set
* add yaml dependency
* Always require sdk-version, and emit warning on mismatch with env var
* More explicit about where override comes from
* Add packaging regression test
* generate choice instances for cross sdk dependencies
* add line to test cross-sdk exercise of choice
* revert to naming choice in GenerateSimpleDalf as "NotChoice"
* remove filter/special-handlimg for "Archive" choice
* dont assume a tycon with the same name as a choice; instead use type from chcArgBinder
* error if cant find the template datatype def
* naming
* avoid intercalate followed by re-split on "." in module name
* GenerateSimpleDalf: have two choices: one of record type, one of variant type
* Update Upgrade.hs
* added some linebreaks, spaces for readability
* address cosmetic comments
* use Sdk. prefix as necessary so tests work
* mollify hlint
* fix: daml-assistant/integration-tests
* Change title of release notes tab
This suggests that a new release is available, but not necessarily
installed. We do this for the case that the extension is updated
(which triggers the tab to open) but the actual SDK is not.
* Keep 'New'
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* MVP for a daml2ts codegen
This PR adds an MVP for a codegen for TypeScript.
Given a DAR, daml2ts replicates the structure of the serializable type
definitions in it as TypeScript type definitions following our JSON
representation of DAML-LF types. It also adds decoders for all these types,
which can be used to check where an arbitray JSON value has the given type.
Finally, daml2ts also produces one JavaScript object for each template, which
reflects the type information of that template.
All produces objects implement some interfaces defined in a TypeScript
library currently called `@digitalasset/daml-json-types`. This libary is not
yet uploaded to NPM but rather included in the `tests/ts/daml-json-types`
directory. This libary also contains the JSON decoders for all of DAML-LF's
builtin types.
There are quite a few limitations right now. Most notably, variant and enum
types are not properly typed right now but rather gradually "typed" as
`unknown`. We also don't support nested `Optional`s, the `Numeric` type or
sum-of-product types in DAML. These issues are tracked in #3518.
There is currently one test. It takes a very simple DAML model, generates
the TypeScript for it and checks that it compiles and contains no linter
warnings/errors. Proper integration tests against the JSON API will follow.
* Address @cocreature's comments
* Make test work on Windows
* language: make only needed packages visible for cross sdk imports
This gives us more control over what packages are visible in the package
database during cross sdk imports and makes sure we're not accidentally
picking up the wrong one.
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* deal with evil dalf imports
* Move packaging tests to a separate test suite
Originally, we had these as part of the integration test suite since
`bazel run damlc build` couldn’t locate `ghc-pkg` but that has been
fixed for a while.
Moving it to a separate test suite speeds things up as the integration
tests are rerun very often and also makes development much more
convenient since the new test suite supports `-p` properly to filter
to specific tests.
For now, I’ve left the upgrading tests as part of the integration
tests. I expect that we probably want to factor those out to another
test suite as well.
* fix warnings
Previously we opened file FooX before FooX+1 which we import from FooX
so there is a race condition if we try to resolve imports in FooX fast
enough. Now we open them in reverse order to fix this.
The other option would be to write all files to disk before opening
them but given that we can easily avoid this here, I’d rather keep
things in memory.
It has grown large enough that by now it definitely deserves to be in
its own module.
This PR does not change any of the actual code, it simply moves it around.
* language: fix for tuple types in source generation
We correct the source generation from dalfs containing TupleN data
types. A test is added to the integration tests of daml assistant and
the `generate-src` command is improved to make this test possible.
* check that tuple type comes from DA.Types
* check that compilation succeeded
* pattern match golf
* check for tuples between 2 and 20
This change allows to only import `module A` from the instances package
if data dependencies are present instead of `module A` and `module
AInstances`.
* Add toAnyContractKey and fromAnyContractKey
This is necessary to add exerciseByKey to DAML triggers.
* Switch to proper ghc-lib release
* Remove unnecessary filter
* Bump timeout because macos is terrible
* bazel fmt
I didn't really read the code around the FIXME. After spending a few moments
on it, I'm now convinces the current behaviour is the right behaviour. I
can't think of a short way to explain it in a comment without inflicting my
my previous confusion on others. Thus, I just remove the FIXME.
* No longer depend on "@bazel_tools//tools/jdk:jar"
To avoid the following warnings
```
WARNING: /home/aj/.cache/bazel/_bazel_aj/c1e06e2358666d118d0ae50e2d32c25d/external/bazel_tools/tools/jdk/BUILD:124:1: in alias rule @bazel_tools//tools/jdk🫙 target '@bazel_tools//tools/jdk:jar' depends on deprecated target '@local_jdk//:jar': Don't depend on targets in the JDK workspace; use @bazel_tools//tools/jdk:current_java_runtime instead (see https://github.com/bazelbuild/bazel/issues/5594)
```
* Targets and files should not share names
To avoid the warning
```
WARNING: /home/aj/tweag.io/da/da-master/compiler/damlc/tests/BUILD.bazel:316:1: target 'simple-dalf.dalf' is both a rule and a file; please choose another name for the rule
```
* bazel: 0.28.1 --> 1.1.0
* bazel-watcher sha256
* Fix missing line in patch
* proto_source_root --> strip_import_prefix
See https://github.com/bazelbuild/bazel/issues/7153 for details.
* Update rules_nixpkgs
Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```
and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
Traceback (most recent call last):
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```
* Update rules_scala
* .proto has been removed, use [ProtoInfo] instead
See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library
* python3_nix add nix_file attribute
To avoid the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```
* rules_haskell unnamed string.split(_, maxsplit = _)
The keyword argument may no longer be named.
* string.replace(_, _, maxsplit = _) may not be named
* Move proto sources from deps to data
Fixes
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```
* Define sha256 for haskell_ghc__paths
Bazel 1.1.0 fails on missing hashes.
* Disable --incompatible_windows_native_test_wrapper
* //compiler/daml-extension don't modify sources
Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.
* bazel-genfiles --> bazel-bin
The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651
* Mark dev_env_tool repository rule as configure
See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule
* Move data deps into data attribute
* Mark dev_env_tool as local = True
* Manually fetch @makensis_dev_env
When buidling simple project that has our favourite large project as a
dependency, this decreased
- total allocations from 63GB to 57GB
- run time from 34.0s to 31.5s
* language: introduce data-imports
Right now the user experience for importing dalfs and dars from
different sdks is quiet confusing. This PR tries to solve this. We add
an additional field `data-imports` to daml.yaml. These imports can come
from different SDK's and we will generate interface files containing the
data types and their Template instances.
This also simplifies the migration command, as it now always imports the
respective packages as `data-imports`.
Given that I initialy messed this up during development (that’s where
the `reverse` in setting up the typecheck env comes from), it seems
like a good idea to have a test for this :)
* Add support for on-disk incremental builds in damlc build
* Normalise file paths of internal modules because Windows
* stop stealing my $s hlint
* Apparently jars are also called exe
* Address review comments
* Bump to proper ghcide revision
* Display release notes using webview
* Use const and fix string
* Check for version upgrade before showing release notes
* Changelog entry
* Use node-fetch instead of web-request
* Remove spurious state update
* language: cross sdk dalf/dar imports
The final piece for cross sdk imports. With this PR we can import the
data types of packages and dalfs that were created with different sdks.
This is done by generating interface files from dalfs and an 'instances'
package that contains the template instance definitions of template data
types. The instances itself are defined via the `external` keyword,
which is inlined to proper daml-lf instance definitions given in the
respective dalf package.
We test that cross sdk imports work by importing the `simple-dalf` in
the daml-assistant integation tests and running a scenario.
* Allow data A = A by prepending DamlEnum$ to type name.
* Single con enums basically work.
* Fix export lists for single constructor enum types.
* Revert "Fix export lists for single constructor enum types."
This reverts commit 7475a3dfbe3531d3ef62fdbcfe64c01a9e22d7af.
* Switch to a "stupid theta" approach
* Clean up enum type preprocessor
* Run enum preprocessor on generated code.
* Add daml-docs golden test for single constructor enums
* s/genPreprocessor/generatedPreprocessor/g
* Update copyright header
* Update release notes
* Remove unnecessary OverloadedStrings
Currently we hide modules, for which we don't define an alias, when we
specify one alias for *one* module. With this change, all modules
without aliases are still exposed.
* language: refactoring of iface file generation and package db setup.
This is a refactoring of the damlc part that creates the package
database and the code generation for interface files. This is a
preparation for the cross sdk imports.
We also add an internal command to damlc to generate generic instances
code and use simple copying via {..} in the migration command. An
additional test checks that migration via generics still works.
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/Upgrade.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* clearer description for generate-gen-src
* updated documentation
* correct copy/pasta mistake
* added a comment on different build options in migration command.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* resolve dalf paths from dar manifest
* added a comment on different headers in upgrade modules.
* removed monoid instance for ExtractedDar
