* Use package metadata instead of file names to infer unit ids
This PR adds a function that abstracts over whether we get metadata
from a filename (< 1.dev) or directly from the LF metadata.
There is more work to be done here, in particular, I want to clean up
the hacks around daml-prim/daml-stdlib but I’ll leave that for a
separate PR.
changelog_begin
changelog_end
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Util.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Refactor getUnitId
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
* Graceful error handling in `daml repl`
This PR changes `daml repl` to handle errors (parse errors, type
errors, unsupported statement errors, script errors) gracefully
and just emit an error message instead of tearing down the whole
process.
This gets the repl into a state where I think it’s sufficiently
user-friendly to be released (obviously there are tons of potential
improvements). The only thing missing before I’m comfortable
mentioning this in release notes and uninternalizing it are docs.
If you think there is something crucial that needs to be addressed
before, let me know.
changelog_begin
changelog_end
* why is windows
As mentioned in the title, this is still very experimental and needs
more work before we want to advertise it. However, the code is in a
somewhat reasonable shape, there are tests and I think even in the
current state it is already useful. Also this PR is already getting
very large so I don’t want to hold off much longer before merging this.
It is included in the SDK but hidden from `damlc --help` and `daml
--help` until the most pressing issues are addressed (primarily around
making sure that it doesn’t just shut down if you have a type error
and better error messages in general).
changelog_begin
changelog_end
This PR adds the necessary infrastructure to produce package metadata
in `damlc`.
For `damlc build` this works exactly as expected. There are a few edge
cases where we don’t have names and/or versions (namely scenarios,
damlc compile and damlc package). We don’t really care about the
metadata for those anyway, so I’ve just set it some default value.
changelog_begin
changelog_end
* Refactor handling of package names and versions
This is a preparatory refactoring PR in preparation for propagating
package metadata into DAML-LF. There are no actual changes in here.
Primarily the changes consist of 3 things:
1. In options, we split the `optMbPackageName` field which previously
contained the unit id into `optMbPackageName` and
`optMbPackageVersion`.
2. We use newtypes for names and versions and try to keep them pretty
much everywhere (the only place missing is `splitUnitId`, I’ll do that
separately).
3. We use `UnitId` where we want `name-version`.
As was probably to be expected, this surfaced some minor issues. They
are pretty much exclusively in debugging or “internal” commands so
I’ve mostly just added notes/todos.
changelog_begin
changelog_end
* cry about applicativedo
`addDays d (- r)` looks a bit confusing and it’s not obvious that
negative numbers even work.
changelog_begin
- [DAML Standard Library] Add `subtractDays` to the DAML Standard Library.
changelog_end
* Add package metadata to DAML-LF proto and the Haskell AST
This adds package metadata (currently only the package name and
version) to DAML-LF and the corresponding Haskell ASTs. This is useful
for debugging and “codegens” (typescript, damlc dependencies, …)
This PR does not yet add it to the Scala side or change the compiler
to actually produce this metadata.
Part of #4412
changelog_begin
changelog_end
* Address review comments
* Add type-level strings in DAML.
This PR adds a `PromotedText` stable package, with `PromotedText` type, which is used to encode type-level strings from DAML into DAML-LF. The reason for this is to preserve the `HasField` instance argument. This PR adds a test that `HasField` is succesfully reconstructed incontexts, during data-dependencies, which wasn't possible before.
changelog_begin
changelog_end
* adresss comments
* fix overly specific tests
* Use KeyHasher to serialize contract keys in kvutils
- Use Value instead of VersionedValue in GlobalKey as the versioning does not make sense here
and may be misleading as the a value with a different version but same meaning would still
be the same key.
- Relocate the KeyHasher to ledger-api-common so kvutils can use it (otherwise cyclic dependencies)
- Replace storing of the contract key as a VersionedValue with the hash produced by KeyHasher.
This is backwards incompatible. A compatible option would require us to query the key with both
the old way and the new way which is unattenable. We're making a calculated breaking change.
CHANGELOG_BEGIN
- [DAML Ledger Integration Kit] Serialize contract keys using a hash instead of the value in kvutils.
This is a backwards incompatible change to kvutils.
CHANGELOG_END
* Use proper hasher for contract keys and not KeyHasher
- Use Hash.scala, not KeyHasher.scala.
- Add hash to GlobalKey as we want the hash to be computed from the inside.
The use of KeyHasher will be later deprecated and replaced by this.
* Use "sealed abstract case class" trick instead of private ctor
and rebase fix
* Revert change to unsupported value version decode error
* Reformat code
* Add kvutils changelog entry and bump the version
* Make DAML's (>>) operator lazy in its second argument
Currently, when you write
```haskell
do
Left "wanted"
error "unwanted"
```
your computation fails with an exception coming from the `error "unwanted"`
call. However, `do`-notation suggests that you actually never get there but
rather bail out at the `Left "wanted"` line. The cause of this mismatch is
that GHC desugars the code above to
```haskell
Left "wanted" >> error "unwanted"
```
and since DAML is strict, we evaluate `error "unwanted"` _before_ evaluating
the application of `(>>)`.
This PR solves the problem by rewriting all expressions of the form `A >> B`,
and hence those of the form `do A; B` as well, to `A >>= \_ -> B`. This gives
the desired semantics with `(>>)` being lazy in its second argument.
However, these semantics only make sense for `Action`s (aka monads). Thus, we
need to restrict the constraint on `(>>)` from `Applicative` to `Action`.
(This is in line with what Haskell does.)
Moreover, the `Action` instance of the `Validation` applicative would lead
to undesriable behavior when using `do`-notation. Thus, we also drop this
`Action` instance and hence force the usage of the `ApplicativeDo` language
extension when using `do`-notation for `Validation`.
CHANGELOG_BEGIN
- [DAML Stdlib] Restrict the ``(>>)`` operator to instances of ``Action``
and make it lazy in its second argument. This gives expressions of the
form ``do A; B`` the desirable semantics of only running ``B`` when
``A`` is a successful action.
- [DAML Stdlib] Remove the ``Action`` and ``ActionFail`` instances for
``Validation`` in ``DA.Validation``. Please enable the ``ApplicativeDo``
language extension if you want to use ``Validation`` with ``do``-notation
and replace ``fail`` with ``DA.Validation.invalid``.
CHANGELOG_END
* Address @cocreature's remarks
CHANGELOG_BEGIN
CHANGELOG_END
We forgot to import the module that defines the duplicate instance
which causes problems if this instance is required for something else,
e.g. because it is a superclass of another instance.
This fixes the issue surfaced by the removal of the Action instance
for Validation in #4552.
changelog_begin
changelog_end
Shouldn’t really make a difference due to laziness but at least it
makes it explicit if we need to decode the archive to the AST or just
need to get the package id.
changelog_begin
changelog_end
On a simple quickstart project we spend about 0.7s in
createProjectPackageDb (all of it in the `withDamlIdeState`
part). This slows down our packaging tests significantly so for now
I’ve added some shortcircuiting if we don’t have any deps and
data-deps. We might want to find a nicer solution here in the
future but the whole code for setting up the package db is going to
move a around a fair bit so I don’t think it makes sense to do this
right now.
changelog_begin
changelog_end
I don’t have a great test case to demonstrate the problem but
`contexts = contexts + (ctxId -> ctx)` is definitely not an atomic
update which means that inserts can get lost.
changelog_begin
changelog_end
I’ve failed to reproduce this locally so far but on CI we sometimes
get the DLint warning that `True == True` is redundant before we get the
parse error which results in the tests failing. There is no reason why
we need to use `True == True` here anyway so the fix is to simply
replace it by `True`.
changelog_begin
changelog_end
If we get interrupted after updating our local view but before doing
the actual context update on the server side bad things happen. To fix
this, we first clone the context then update the clone which avoids
this and is a bit nicer than throwing an `uninterruptibleMask` around
updating the local view and sending the updateCtx gRPC request.
fixes#4525
changelog_begin
changelog_end
* Fix cross-sdk typeclass reexports in data-dependencies
Sorry for the somewhat large PR, the intermediate states sadly just
change error messages but don’t allow me to test the things they
enable (at least not without considerable effort).
There are a few things this PR fixes:
1. We did not properly expose `dependencies` while compiling
`data-dependencies`. Now they are mapped under a Pkg_$pkgId prefix.
2. We did not pass on `dependencies` to the LF typechecker and
various other places that tried to look up LF packages.
3. We did not filter out duplicate instances, e.g., `daml-prim`
defines `Eq` instances for `Either`. If we define that twice, we
will get an error.
4. The use of Generic in DA.Upgrade doesn’t work
(https://github.com/digital-asset/daml/issues/4470). For now,
I’ve added a somewhat crude hack to filter out typeclasses of a
given name. We should replace this by checking for `Erased`
but that is not yet used in 0.13.51 which I use in the test.
For the test, I’ve checked in a DAR built on 0.13.51 with DAML-LF
1.dev. Not sure if checking in DARs is a reasonable thing to do in the
long term but for now, this is small enough and let’s us move forward
with this.
changelog_begin
changelog_end
* Document envDepInstances
* Document TUnit match
* Update docs for stub binds
* Cleanup and document class rewrites
* Report assistant commands and errors via a logger.
CHANGELOG_BEGIN
CHANGELOG_END
* Hook up daml-assistant to a GCP logger.
* fix test case
* fix more tests
* Check opted in status in assistant.
* Anonimize args that have unexpected characters.
* More agressive anonimization
* add missing containers dependency
changelog_begin
- [DAML Standard Library] Add a ``subtract`` function which is useful
as a replacement for sections of ``(-)``, e.g., ``subtract 1`` is
equivalent to ``\x -> x - 1``.
changelog_end
This is a spin off from my fixes for making data-deps work with
typeclasses cross-SDK.
We only have the interface files for the main dalf so it doesn’t
really make sense to extract the other dalfs. The current behavior of
extracting all dalfs results in them being picked up by
`GeneratePackageMap` even if GHC doesn’t know about them which causes
issues in other placse.
I’ve adapted the collision check to check for transitive dependencies
when creating the project db.
changelog_begin
changelog_end
The string needs to include the full flag instead of only including
--package. Otherwise you get rather unhelpful error messages.
changelog_begin
changelog_end
* Handle multiple constraints on an instance in data-dependencies
Previously we translated `instance (Foo a, Foo b) => Foo (a, b) where`
into `instance Foo a => Foo b => Foo (a, b)` which is a syntax
error. We already did this correctly for `HasField` so this PR mostly
just shuffles things around to always use the non-HasField specific
parts.
changelog_begin
changelog_end
* Prefix fields of DFunHead wtih dfh
This adds a test for running the LSP server directly in the project
directory to make sure that we do not need to build first in that case
and that no --package flags are necessary.
changelog_begin
changelog_end
* Translate unsupported kinds to a special Erased type
This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.
changelog_begin
changelog_end
* Switch to HashSet/HashMap for NormalizedUri/NormalizedFilePath
This matches the change to ghcide in
https://github.com/digital-asset/ghcide/pull/420. Now that we have
optimized Hashable instances it makes sense to use them as much as
possible.
changelog_begin
changelog_end
* debug windows crap
* Fix a bug in vr scenario notifications
* Revert "debug windows crap"
This reverts commit f58fdb92c1.
* attempt to fix windows
who are we kidding, this is not going to work
* Support multiple-packages in `damlc ide`
changelog_begin
- [DAML Studio] You can now open DAML Studio in the root of a
multi-package project instead of opening it separately for each
package. Take a look at the documentation for details on how to set
this up.
changelog_end
There are a few caveats here:
1. You need a ``daml.yaml`` in the root of your project directory. I
think this is somewhat sensible but we should add a warning to VSCode
if you open it in a directory that does not have a ``daml.yaml`` (in a
separate PR).
2. Changes are not picked up accross dependencies. This is a larger
undertaking and given the current setup simply impossible (we don’t
know that the source files of one package belong to the DAR referenced
in the ``dependencies`` field of the other package. We can make this a
bit better by at least detecting that the ``.dar`` has changed but
let’s do that separately.
3. Since ``daml init`` runs once on startup, it will run in the root
directory instead of initializing the package db of the individual
packages. This is fixable but will conflict with #4391 so let’s
address this separately.
I’ve added docs to the daml studio section that explain the caveats.
* Use the proper sdk version in lsp-tests
`fakeDynFlags` has been removed from ghcide. We still need it since we
set up our environment in weird ways and want it to be platform
independent so this PR just copies the definition into `daml`.
changelog_begin
changelog_end
We need to use runActionSync everywhere except for the IDE. Pretty
sure I’ve fixed all cases of those. We also need to use useE instead
of use_ since apparently exceptions shortcircuit shake meaning that
runActionSync doesn’t actually do sync. I’ve only fixed the cases that
we hit in the tests so we probably should do a more thorough pass
after this (I want to get this merged soon since it is quite flaky on
master).
changelog_begin
changelog_end
* Added checks for unit id conflicts.
changelog_begin
changelog_end
* Only report conflicting keys
* lint
* fix indentation
* Show conflicting package ids also
I’ve spend several hours trying to get them to fail on Linux and
Windows and have failed completely (apart from making my machine roun
out of memory, thanks crashplan). I also inserted explicit delays to
make the failure more often frequent without any sucess (at making it
fail). I don’t really have a satisfying explanation of why this is
gone now. My best guess is that the `ghcide` upgrade fixed this since
it included some fixes and changes around how we kill and start the
shake process.
fixes#4328
changelog_begin
changelog_end
This adds the NodeInfo trait to compute the informees of a node and the
required authorizers. This has been separated into its own trait as this
computation is also required (externally) for serialized transaction nodes
in a context where the full deserialization of the node would be prohibitively
expensive.
While doing this change and cleaning up transaction authorization it was
discovered that top-level fetches (which can only be created from scenarios)
did not include the submitter as a witnessing party.
CHANGELOG_BEGIN
[DAML SDK] Fix computation of witnesses of top-level fetch nodes in scenario results ("known to").
CHANGELOG_END
Previously, we did not send any message when users simply clicked away
the telemetry popup. Now we send a special message similar to the
opt-out message which only includes the machine id.
I’ve also changed the opt-out message to include the machine id.
changelog_begin
changelog_end
* Use a safer safeToReexport
This is much safer than the approximation from last time. The only downside is
introducing a dependency between data dependencies and our type checker,
but that seems safer than having two versions of `expandTypeSynonyms`
floating around (and perhaps this dependency is something we would end
up adding anyway).
changelog_begin
changelog_end
* Add own package to extPackages.
* Use mkTForalls
* simplify mkTForalls
* Re-export typeclasses in data-dependencies...
... whenever possible.
The `safeToReexport` over-approximation is simple but troubling, so
I'd like to get it fixed soon.
The inputs to generateSrcFromLfPkg (and the genInstances one) were
getting quite hairy, so I decided to pass a record instead.
changelog_begin
changelog_end
* Add a test
* Move generateSrcPkgFromLf to its original place
* Better docs and make envQualify more clear
* compute dependency ids in generate-src command
The packaging tests are already one of our slowest test suites and
damlc test takes quite a while since it has to spin up the scenario
service.
We already have tests for damlc tests so this PR moves the tests from
the packaging test suite that are specific to `damlc test` to those
tests which should balance things a bit better.
changelog_begin
changelog_end
Based on a comment by @cocreature in the last PR, this PR changes the
generation of typeclass instances to empty stubs, instead of adding
stub methods as well.
changelog_begin
changelog_end
Sadly my fix in https://github.com/digital-asset/daml/pull/4325 didn’t
seem to work (despite fixing the issue in my tests) so let’s mark this
as flaky until we figure it out.
changelog_begin
changelog_end
We have seen a couple of flaky tests runs on CI in the packaging test
caused by this. It is easy to reproduce if you add a delay in
diagnosticsLogger before printing the diagnostics which makes this
failure reproducible. With this patch, the diagnostics are always
printed even with a delay.
changelog_begin
changelog_end
* Remove mkOptions and defaultOptionsIO
`mkOptions` is a terrible API and this PR burns it with lots of 🔥.
The only thing that I didn’t simply move around is the validation
logic in `mkOptions` which validates that some directories
exist. Given that I’ve never actually see this be useful and the
options to set these directories (package dbs, import paths) are
internal I don’t think we actually gain anything from this.
I’ve also killed defaultOptionsIO which was a very confusing wrapper
around `mkOptions` that in addition to calling `mkOptions` also
enabled DLint. This was only used in tests, so I’ve enabled DLint in
the tests that need this.
changelog_begin
changelog_end
* newtype version header
* Generate mod refs at the same time as conversion
* Small change
changelog_begin
changelog_end
* lint
* Get rid of secondM
* Switch to Writer.CPS
* doc typo
* Added a partition function to prelude.
* Fixed partition function documentation.
* CHANGELOG_BEGIN
- Added `partition` function to prelude.
CHANGELOG_END
* Fixed partition documentation.
`initPackageDb` assumes that `Options` has not yet been initialized
and calls `mkOptions` itself. Each call to `mkOptions` appends the LF
version to the package db dir which means that calling it twice as we
did in `execIde` results in pkg dbs of the form dir/1.7/1.7 which is
obviously not what we want.
This is just the fix, I’m sufficiently annoyed by this now, that I’ll
spend some time tomorrow to kill mkOptions completely but for now this
at least fixes the SDK on master.
The reason why we didn’t catch this in our tests is that the package
dbs are located slightly differently via Bazel runfiles as they are
located in the final release tarball which ended up not breaking this
in our LSP tests.
changelog_begin
changelog_end
Unfortunately, downgrading `scalafmt` resulted in files that would be
reformatted to invalid code, because lines would be merged but trailing
commas would be left in.
I've manually run it everywhere and fixed these issues so people don't
have to fix them as they go.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Refactor modrefs out as a type.
This is in preparation for moving to a monadic style.
changelog_begin
changelog_end
* Update compiler/damlc/daml-compiler/src/DA/Daml/Compiler/DataDependencies.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Currently, the DAML language server use for DAML Studio crashes when the
`build-options` section of `daml.yaml` contains flags like `--package`.
This PR fixes this issue by allowing most of the flags for `damlc build`
for `damlc ide` as well.
CHANGELOG_BEGIN
- [DAML Studio] Allow all command line flags for ``daml build`` for
DAML Studio as well.
CHANGELOG_END
* Bazel test for trigger scenario
* daml-triggers: Allow testing trigger rules in scenarios
CHANGELOG_BEGIN
- [DAML Triggers - Experimental] DAML triggers can now be tested in
scenarios. Specifically, a trigger's ``rule`` can be executed in a
scenario and assertions performed on the emitted commands.
CHANGELOG_END
* Allow assertions on create commands
CHANGELOG_BEGIN
* [DAML stdlib] Add `CanAbort` instance for `Either Text`.
CHANGELOG_END
* Add convenience to construct ACS for testRule
* Add assertions for exercise and exerciseByKey
* fix assert message
* Test assertExercise(ByKey)Cmd
* unpackCommands --> flattenCommands
* Add API documentation
* Document that command ids start from "0"
* generalise command assertions to CanAbortm
* export ACSBuilder type
* Better haddocs for trigger command assertions
* explicit let
* ./fmt.sh
* Fix runfiles on Windows
* Add reference to Bazel issue
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
fixes#3436
changelog_begin
- [DAML Compiler] ``damlc test`` now initializes the packagedb automatically which
means that it will work on projects that declare custom
``dependencies`` in ``daml.yaml`` without having to call ``damlc
init`` first.
changelog_end
* Preserve class method names during LF conversion.
This PR changes the LF conversion to pick names for dictionary
fields that will be easier to reconstruct as typeclasses
later.
For superclasses this uses "s_1", "s_2", "s_3" and so on,
and for methods this uses "m_foo", "m_bar", "m_baz" and
so on. (Is this enough C++?)
In the future we might want to distinguish between methods
that are mandatory and methods that are optional ... but
I think this should be good enough for now.
This PR also adds a test.
Fixes#4216
changelog_begin
changelog_end
* lint
* Run test in LF 1.7 as well
* Add test for superclass field.
They have been migrated to `ghcide` in
https://github.com/digital-asset/ghcide/pull/388 which is a much more
sensible place for them to live given that the code is in that repo.
changelog_begin
changelog_end
* Fail when a data-dependency fails to compile
Previously, we emitted errors but just kept on going which sometimes
kinda works but is certainly not the right thing to do.
Now we just crash and abort compilation.
changelog_begin
changelog_end
* why is windows
* undo crap
This fixes#4114 and cleans up the situation around
`data-dependencies` and `dependencies` as described in #4218.
There is still more work to be done here mostly around ironing out all
the edge cases and producing useful error messages instead of
silentely doing the wrong thing but I’ll leave that to a separate PR.
To test this, I’ve fixed the packaging tests to no longer deduplicate
package ids (which means they would return the wrong number if
daml-prim ends up in there twice) and I addded a test where we have 3
projects:
- `lib`
- `a` which depends on `lib`
- `b` which depends on `lib` via `dependencies` and 'a' via
`data-dependencies`.
changelog_begin
changelog_end
* Converting functions in data dependencies.
changelog_begin
changelog_end
* Add NoOverloadedStrings to the set of extensions in generated packages.
* Just use the fn as its own right hand side
* Restore order of package map (not that it makes a difference)
* Adjust imports
* Weird lint but ok
* Make the test pass somehow
* Dont preprocess enums in GHC.Prim
* Preprocess enums everywhere, and add mod ref as needed.
* Revert preprocessor changes
* Dont expose old-style typeclasses
* Dont convert newstyle typeclasses temporarily either
* Add test for function importing
* Support DAML-LF type synonyms in scala.
CHANGELOG_BEGIN
CHANGELOG_END
* dont create synonymns in GenerateSimpleDalf
* extend DAML-LF parser to support type synonyms
* test: expand type synonyms correctly
changelog_begin
- [DAML Compiler] Choices marked explicitly as `preconsuming` are now equivalent to a non-consuming choice that calles `archive self` at the beginning.
changelog_end
Co-authored-by: Shayne Fletcher <shayne.fletcher@digitalasset.com>
* Remove damlc.jar
We never advertised or published this externally and our only internal
user has moved off this months ago already.
changelog_begin
changelog_end
* Remove dependency from navigator test lib on damlc jar
This makes me a :sadpanda: but until we have figured out what is going
on, this seems better than forcing manual restarts.
changelog_begin
changelog_end
* Handle unserializable scenario result types
Previously, the conversions made some attempt at guarding against this
by matching on SPap and PClosure. While it would be possible to extend
this to match on STypeRep, this doesn’t actually fix the issue since
this can be nested, e.g., you can have a record with a field that is
an SPAP.
This PR changes this to simply catch any errors thrown from
`toValue`. While this feels a bit ugly, I think it’s a reasonable fix
for now.
changelog_begin
- [DAML Studio] Scenarios with unserializable result types no longer
crash the scenario service.
changelog_end
* Only run test on DAML-LF >= 1.7
* Remove intermediate PackageImport type
We can just reuse GHC’s PackageFlag directly which simplifies things a
bit.
changelog_begin
changelog_end
* windows is bad
* meh
* foobar
* why
To differentiate between "data X = X" which is translated to a DAML-LF
enum and "data X = X {}" which is translated to a DAML-LF record, we
add a datatype context with GHC.Types.DamlEnum. For
`data-dependencies` this needs to point to Currentsdk.GHC.Types.
changelog_begin
changelog_end
* Use GHC’s syntax for the --package flag
The previous syntax was just based on the Read instance of a 3-tuple
which was pretty much unusable. This changes it to GHC’s much nicer
syntax.
I deliberately did not add a changelog entry for this, since this
flag isn’t something that we have documented at all and it is
currently only useful in combination with data-dependencies.
fixes#4126
changelog_begin
changelog_end
* Fix hlint
* get it to compile 😇
* shut up hlint
* Remove the key field from pkg.conf files
We never use this and it only a legacy field that is deprecated in
favor of id in GHC.
There is also a minor change in the docs around ExposePackage which
should at most affect error messages.
changelog_begin
changelog_end
* Better error messages
The packaging tests have grown a fair bit recently and seem to hit
timeouts sometimes. We might want to split it up into multiple tests
suites but for now this should help.
changelog_begin
changelog_end
The current code has gotten out of sync with the changes to template
desugaring and is therefore simply broken. In addition to that, we
don’t need this once we can use typeclass instances in combination
with data dependencies.
Initially, we just disabled this instead of removing it since we
thought it might still be useful for older packages before typeclasses
are desugared to type synonyms. But we already have one way of
handling those as evidenced in the tests for the DAVL and once we can
call functions from older packages (which works regardless of whether
they have been built before or after the changes to how typeclasses
are desugared), we have another one so I don’t think it makes any
sense to keep this code around.
changelog_begin
changelog_end
* Refactor damldocs
* Refactor damldocs.
changelog_begin
changelog_end
* More refactoring
* Update copyright headers.
* Add export list for Templates
* Add export list to .Util
* added a little ^
* Test data-dependencies for more LF version combinations
This PR extends the cross-LF version tests to run for all
combinations where depLfVer >= targetLfVer.
changelog_begin
changelog_end
* Fix comments and add TODO
This PR fixes a minor issue where we were including more references
than we need which causes problems on older SDKs where GHC.Prim is not
a stable module since GHC.Prim cannot be imported (it’s builtin GHC
magic).
More importantly, this PR adds a test for cross-SDK data-dependencies
against the DAVL DAR which is built with SDK 0.13.32 before we started
making all the changes. The test also includes a slightly hacky but
mostly reasonable way of actually using the templates instead of just
the data types from the DAVL DAR which and a scenario to test this.
changelog_begin
changelog_end
This fixes the ZIP modification times in all DARs to a specific
value (1980-01-01) whereas they used the current time before. This
both gives us the nice property that not only our DALF builds but also
DAR builds should be deterministic (and there is a test for this). I
have a suspicion that this could help significantly with build times
and avoid rerunning half of the Scala tests on a change to damlc that
should not change the DALFs.
changelog_begin
- [DAML Compiler] The modification times in a DAR are now fixed to a
given value which makes the output of ``daml build`` deterministic
in single-threaded mode (which is the default).
changelog_end
This PR fixes a whole bunch of bugs when using data-dependencies on a
package produced by damlc rather than the handcrafted simple-dalf.
The end result is that we are able to build a package with
`--target=1.6` and use it as a data-dependency from a package with
`-target=1.7`. While the test that I’ve added for this is not
cross-SDK it is arguably even trickier since we cannot rely on the
unit id of daml-stdlib changing the version in the unit id.
The changes all fall into one of the following two categories:
- Special handling for daml-prim and daml-stdlib to avoid collisions
with the one from the current SDK.
- Special treatment of stable packages to avoid generating new code
for those.
- Remove a couple of filters for daml-prim since we do need to include
daml-prim from a data-dependency if it is a different version.
This is definitely not perfect and there are a bunch of rather hacky
parts in here but it seems like a good intermediate step (and it has a
test :))
changelog_begin
changelog_end
This was accidentally set to 1.7 before for no reason. Note that this
simply meant that we did not use the stable module when compiling with
--target=1.6 since we filter out newer stable packages.
Nevertheless, I’ve also added a test that verifies that we do not
accidentally introduce dependencies on packages in newer LF versions.
changelog_begin
changelog_end
* Try to improve the docs on DA.Internal.Template.Functions a little.
* More docs.
changelog_begin
- [DAML Standard Library] The ``Template``, ``Choice``, and
``TemplateKey`` typeclasses have been split up into many small typeclasses
to improve forward compatibility of DAML models. ``Template``,
``Choice`` and ``TemplateKey`` constraints can still be used as before.
changelog_end
* Example typo
* Bump ghcide to include bugfix
This includes a fix for a bug in completions which we introduced in
the latest ghcide bump which in turn broke the DAML IDE completely.
changelog_begin
- [DAML Studio] Fix a bug in completions that caused DAML Studio to
stop responding after the first completion was requested.
changelog_end
* Use ghcide from master branch and simplify test
* Patterns and test for new desugaring
* working on new template conversion
* Update ghc-lib
changelog_begin
changelog_end
* shut up hlint
* Update desugar stdlib
* update test
* remove unuseful templatebinds
* Add implicit qualified GHC.Types import
* Add missing primitives
* Remove chaff
* update comments
* Remove patterns that dont seem useful anymore
* Capture key data in template binds
* Dont make TypeRep/ToAny/FromAny classes conditional
* Remove some unnecessary TODOs
* Generate the template definition
* Remove new template desugaring test
* Fix jq query
* Rename makeDesugarDFunProjection to useSingleMethodDict
* Let TTypeRep and TAny be TUnit in primitives.
* Fix damlc visual wrt the new desugaring
* Fix visualization tests in shake test suite
* Fix damldocs
* Drop envTemplateKeyData
* Use the new ghc-lib release
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Allow invoking generate-src on the main dalf of a dar
Given that daml build outputs a .dar file this is much more convenient
for testing against actual build outputs of damlc rather than
handcrafted .dalf files.
The behavior matches the one of `damlc inspect` in that it detects
whether it works on a .dar or a .dalf based on the file ending.
changelog_begin
changelog_end
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
The generation of Template instances will break horribly with
https://github.com/digital-asset/daml/pull/4030 and we don’t actually
want to use this anymore since we want to reuse old class
instances. There are some cases where we might still need or want this
in the future (see the inline comment), so for now, the code for this is still left intact and
only disabled.
changelog_begin
changelog_end
This PR adds a bunch of top-level helpers that take care of building
up the GHC AST.
I’ve also briefly played around with not using the GHC AST at all but
that doesn’t seem to really help simplify things so for now, I think
this is the better approach
changelog_begin
changelog_end
Previously, we exposed those functions from Prelude which really
doesn’t make any sense given that they are intended for the internal
implementation of daml-prim and daml-stdlib.
changelog_begin
changelog_end
* Respect build-options if project-root is used
changelog_begin
- [DAML Compiler] The ``build-options`` field from ``daml.yaml`` is
now also respected if ``--project-root`` is used.
changelog_end
* address review comments
* CHANGELOG_BEGIN
Type-check type synonyms.
CHANGELOG_END
* placate HLint
* comments
* Add an example that requires the check in kindOf
* check types containing syn-apps are well formed even when there is no expression of that type
* show type mismatch error after synonyms are expanded
* typeOf calls expandTypeSynonyms; track vars bound by TForall during expansion
* test interaction of syn-expansion and free-vars; add one bigger testcase
* extend bigger example with pointed typeclass, having functor as a super class
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Send session ping telemetry messages
This PR adds telemetry information that is sent regularly (every 5
minutes) while users are active (they’ve opened, closed or changed a
file). This allows us to track how long user sessions are.
This PR does not include any additional information in the session
ping but we plan to add more in the future.
The ping messages can be found in big query by filtering for
`jsonPayload.type = "session_ping"`.
CHANGELOG_BEGIN
CHANGELOG_END
* add copyright header
As mentioned in the comment, it doesn’t really make sense to include
the SDK version in the DALF names but I’ll address this in a separate
PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Add regression test for unstable serializable types
This test verifies that we only have the serializable types in
daml-prim and daml-stdlib that we expect and don’t introduce new ones
by accident.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix windows
* Ignore whitespace because windows
* optional is dead
All references to Optional are already mapped to the builtin Optional
type in LF but we forgot to filter out the type definition.
CHANGELOG_BEGIN
CHANGELOG_END
* Move most of the remaining serializable types to stable LF packages
The only serializable types left in DAML stdlib after this PR are the
following:
- DA.Upgrade:MetaEquiv
- DA.Random:Minstd
- DA.Next.Set:Set
- DA.Next.Map:Map
- DA.Generics:MetaSel0
- DA.Generics:MetaData0
- DA.Generics:DecidedStrictness
- DA.Generics:SourceStrictness
- DA.Generics:SourceUnpackedness
- DA.Generics:Associativity
- DA.Generics:Infix0
- DA.Generics:Fixity
- DA.Generics:K1
- DA.Generics:Par1
- DA.Generics:U1
- DA.Internal.Prelude:Optional
Ignoring the Generics stuff which isn’t very urgent imho and the
Upgrade stuff which is probably going to change significantly anyway,
this leaves us with the weird Random module, the wrappers around
TextMap which will go away anyway and DA.Internal.Prelude:Optional
which shouldn’t exist in the first place (I’ll address that in a
separate PR).
CHANGELOG_BEGIN
- [DAML Compiler] Move more types from daml-stdlib to standalone LF
packages. The module names for the types have also changed
slightly. This only matters over the Ledger API when you specify the
module name explicitly. In DAML you should continue to use the
existing module names.
- The types from ``DA.Semigroup` are now in a separate package under
``DA.Semigroup.Types``
- The types from ``DA.Monoid` are now in a separate package under
``DA.Monoid.Types``
- The types from ``DA.Time` are now in a separate package under
``DA.Time.Types``
- The types from ``DA.Validation` are now in a separate package
under ``DA.Validation.Types``
- The types from ``DA.Logic` are now in a separate package under
``DA.Logic.Types``
- The types from `DA.Date` are now in a separate package under
`DA.Date.Types`.
- The `Down` type from `DA.Internal.Prelude` is now in a separate
package under `DA.Internal.Down`.
CHANGELOG_end
* Fix serializability of RelTime
* fix daml-docs
* Fix tests
I managed to break our Windows CI twice already due to this so while
it makes me sad, I think hardcoding those files is the more sensible
option for now.
* Move Any wrappers and Archive to stable packages
There are no actual API or functionality changes in this PR but the
logic for locating the stable packages has slightly changed since the
Any wrappers package only makes sense for LF 1.7. To address this, we
simply filter out stable packages for newer LF versions since it
doesn’t make sense to depend on those anyway.
CHANGELOG_BEGIN
- [DAML Compiler] Move ``Archive`` type to a separate DALF.
CHANGELOG_END
* More comments
* Fix java codegen tests
* fix more tests
* Force cache reset on Windows
* Revert "Force cache reset on Windows"
This reverts commit 9f2b7d70b2.
* Fix roundCommercial
fixes#3884
Previously, we relieid on being able to convert to an Int, which
doesn’t work since there are Numeric values that cannot be represented
as an Int. Now, we avoid any conversions to Int which fixes this
issue.
We probably just want to have a primitive for this at some point but
given that this will only be available in newer LF versions, I think
it’s worth working around this for now.
* Move tests to a separate file to exclude LF 1.6
* Update compiler/damlc/daml-stdlib-src/DA/Internal/Prelude.daml
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Apply suggestions from code review
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Update comments
* Update compiler/damlc/daml-stdlib-src/DA/Internal/Prelude.daml
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Move stable types from DA.Internal.Template to a separate module
This is in preparation for splitting them into a separate LF package
with a stable package id.
* Fix visualization
* Fix data dependencies
Originally, this made sense as a flag in Options since we read it
during typechecking to write out interfaces on the fly. But now, we
write this is only used in execCompile so having this be a global
option that is ignored anywhere but in `compile` is rather confusing.
This documents some of the edge cases in the packaging code that I’ve
had trouble understanding so that the next person hopefully has an
easier time with this.
There is also some minor cleanup in this PR.
* Add Eq instances for AnyTemplate, AnyChoice and AnyContractKey
CHANGELOG_BEGIN
- [DAML Standard Library] Add ``Eq`` instances for ``AnyTemplate``, ``AnyChoice`` and ``AnyContractKey``.
CHANGELOG_END
* Add DAML_ANY_TYPE to the CPP guard
* Fix package names in depends field in pkg configs
Previously, we derived this based on the DAR name which breaks if you
use -o with rather confusing error messages. Now, we read it from the
`Name` field in the manifest that we added in
https://github.com/digital-asset/daml/pull/3805.
CHANGELOG_BEGIN
- [DAML Compiler] Fix an issue where transitive package dependencies
resulted in packages not being found, if the DAR name was changed with
`-o`.
CHANGELOG_END
* Fix package dependencies
Currently, we generate the `depends` field in the ghc-pkg config file
based on the file name of the DARs in the `dependencies` field in
`daml.yaml`. This falls apart when you use the -o option to `daml
build` since `ghc-pkg` will complain about missing packages.
This PR adds the name to the manifest so that we can generate the
`depends` field based on that but I’ll leave that change for a
separate PR.
* Update hlint
* Redefine base package to make more sense for daml
Also, elminate lens and attoparsesec packages and groups.
* Don't change the test; ignore the new lints
* More conversion haskell rules to daml
* Move all datatypes out of daml-prim
This moves the remaining two modules DA.Types and GHC.Tuple to
separate LF packages with stable identifiers.
The only data types remaining are the ones for typeclasses which will
disappear once we move this to type synonyms.
CHANGELOG_BEGIN
- [DAML Compiler] The modules DA.Types and GHC.Tuple from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Fix codegen tests
* Fix DarReader test
* Fix kvutils tests
* Fix jdbcdao tests
* Fix hs ledger bindings tests
* Full package name collision check
* Handle type synonyms appropriately
* Better comment
* Make isAscendant case-insensitive
* Document isAscendant and explain case-insensitivity
* Add a package-wide name collision test.
* Refactor packaging logic
This is a first step towards cleaning up the packaging logic and
adding some comments to make it clearer what is going on. There are no
functional changes in this PR.
There is more stuff here that we can and should cleanup but I will
leave that for separate PRs.
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/damlc/lib/DA/Cli/Damlc/Packaging.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Document topological sorting
* Undo requiredE change
Sadly, I haven’t managed to come up with an isolated test case for
this but I’ve tested this on a large internal codebase and it fixes an
issue with damlc build --incremental=yes.
* language: suffix all dalfs dependencies in a dar with the pkgid.
This makes sure that dalf dependencies are not accidentally overwritten
when two packages with equally named dalfs are imported.
* factor out parseUnitId
This is a first step towards making sure that the package ids for
types defined in daml-prim and daml-stdlib don’t change. This PR
mostly adds all the necessary infrastructure for that and moves
GHC.Types and GHC.Prim to make sure it works.
Until data-dependencies are really solid and we have verified that we
no longer have performance issues with an increasing number of Haskell
packages, we still include the source files in daml-prim and then just
rewrite the references.
We will also need to add tests that these packages really have stable
ids but I’ll leave that for separate PRs since this doesn’t make that
much sense anyway until all of the types have moved to stable
packages.
CHANGELOG_BEGIN
- [DAML Compiler] The modules GHC.Prim and GHC.Types from daml-prim
have been moved to separate packages.
CHANGELOG_END
* Track used packages during whole of engine submit
- Introduce MutableCompiledPackages interface
- Add TrackingCompiledPackages that tracks fetched packages
- Make used packages in transaction optional to distinguish between
missing dependencies and empty set of dependencies.
* Reimplement package dependency tracking
- Compute direct dependencies of a package during decoding
- Compute transitive dependencies of a package when adding a package
to engine.
- Annotate the resulting transaction with package dependencies
in Engine.submit.
* Create Ast.Package with proper direct deps in scenario service
While we don't have use for direct dependencies of a package in
scenario service (only Engine.submit needs it), it's better to be
accurate.
This of course overapproximates the direct dependencies.
* Compile a each new package once in ConcurrentCompiledPackages
I’ve gone through references to generic templates in our codebase to
see what else we can drop. In this case, we are still using
coerceContractId so I’ve updated the comment to explain the current
usecase.
We needed this for our generic template hacks. Since they are gone now,
I'm very much in favour of removing this extension since "undecidable"
always sounds a bit scary, even if it is only at compile time.
CHANGELOG_BEGIN
- [DAML Compiler] Don't make `UndecidableSuperClasses` a default language
extension for DAML anymore. If you really need this feature for a module,
you can reenable it using a `LANGUAGE` pragma at the top.
CHANGELOG_END
* Introduce a simpler template desugaring without support for generic templates
This adapts the LF conversion to the new template desugaring
introduced in our GHC fork. The guiding principle is that we use the
typeclasses directly to avoid generating, typechecking and converting
redundant code caused by indirections. I updated the template
desugaring documentation so that is probably a good starting point for
reviewing this.
* Address review comments
* Fix daml doc tests
* Fix data dependency tests
* Switch to new ghc-lib release
Display the actual number the user tried to use instead of 1.2345.
The logic to display the number is not perfect but definitely better than
we have now. If somebody knows a _simple_ way to do this better, please
tell me about it.
Until now, we broke long party names in the table view in the scenario at
spaces. This looked quite ugly.
Now, we don't break them at spaces. At the same time we also moved the CSS
into a file rather than having it in the code to make changes like this one
easier in the future.
This fixes#3642.
This reenables the migration checks. The migration command now only
generates stumbs for conversions instead of using the formerly (generic)
upgrade template.
* Successfully parse rss xml and find latest blog post object
* Show notification with link to blog (on every ide startup...)
* Only show blog post if not seen before
* Formatting
CHANGELOG_BEGIN
- [SDK] Notify IDE users about new daml-driven blog posts
CHANGELOG_END
* Add setting for user to opt out of notifications
* Use await rather than then
* Catch and swallow exceptions
* Another await and minor cleanup
We don't drop full support from the compiler yet but rather ban their use by
adding a check to the preprocessor. We'll remove the actual support as we go
along with fixing the upgrading story.
CHANGELOG_BEGIN
- [DAML Compiler] Make the experimental feature "generic templates"
unavailable. The current implementation is at odds with other, more important
language features still under development.
CHANGELOG_END
* daml-lf: rename Map to TextMap in archive proto
+ in Scala/haskell AST
* a bit more renamming
* Update compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Serializability.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* fix test
* Apply suggestions from code review
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Set Sdk-Version in DAR Manifest to compiler SDK version.
CHANGELOG_BEGIN
- [DAML Compiler] Bugfix: The Sdk-Version field in a DAR manifest file
now matches the SDK version of the compiler, not the sdk-version field
from daml.yaml. These are usually the same, but they could be different
if you set the DAML_SDK_VERSION environment variable before running
``daml init`` or ``daml build``.
CHANGELOG_END
* Fallback to daml.yaml if env var not set
* add yaml dependency
* Always require sdk-version, and emit warning on mismatch with env var
* More explicit about where override comes from
* Add packaging regression test
