* Refactor handling of package names and versions
This is a preparatory refactoring PR in preparation for propagating
package metadata into DAML-LF. There are no actual changes in here.
Primarily the changes consist of 3 things:
1. In options, we split the `optMbPackageName` field which previously
contained the unit id into `optMbPackageName` and
`optMbPackageVersion`.
2. We use newtypes for names and versions and try to keep them pretty
much everywhere (the only place missing is `splitUnitId`, I’ll do that
separately).
3. We use `UnitId` where we want `name-version`.
As was probably to be expected, this surfaced some minor issues. They
are pretty much exclusively in debugging or “internal” commands so
I’ve mostly just added notes/todos.
changelog_begin
changelog_end
* cry about applicativedo
`addDays d (- r)` looks a bit confusing and it’s not obvious that
negative numbers even work.
changelog_begin
- [DAML Standard Library] Add `subtractDays` to the DAML Standard Library.
changelog_end
* Add package metadata to DAML-LF proto and the Haskell AST
This adds package metadata (currently only the package name and
version) to DAML-LF and the corresponding Haskell ASTs. This is useful
for debugging and “codegens” (typescript, damlc dependencies, …)
This PR does not yet add it to the Scala side or change the compiler
to actually produce this metadata.
Part of #4412
changelog_begin
changelog_end
* Address review comments
* Add type-level strings in DAML.
This PR adds a `PromotedText` stable package, with `PromotedText` type, which is used to encode type-level strings from DAML into DAML-LF. The reason for this is to preserve the `HasField` instance argument. This PR adds a test that `HasField` is succesfully reconstructed incontexts, during data-dependencies, which wasn't possible before.
changelog_begin
changelog_end
* adresss comments
* fix overly specific tests
* Make DAML's (>>) operator lazy in its second argument
Currently, when you write
```haskell
do
Left "wanted"
error "unwanted"
```
your computation fails with an exception coming from the `error "unwanted"`
call. However, `do`-notation suggests that you actually never get there but
rather bail out at the `Left "wanted"` line. The cause of this mismatch is
that GHC desugars the code above to
```haskell
Left "wanted" >> error "unwanted"
```
and since DAML is strict, we evaluate `error "unwanted"` _before_ evaluating
the application of `(>>)`.
This PR solves the problem by rewriting all expressions of the form `A >> B`,
and hence those of the form `do A; B` as well, to `A >>= \_ -> B`. This gives
the desired semantics with `(>>)` being lazy in its second argument.
However, these semantics only make sense for `Action`s (aka monads). Thus, we
need to restrict the constraint on `(>>)` from `Applicative` to `Action`.
(This is in line with what Haskell does.)
Moreover, the `Action` instance of the `Validation` applicative would lead
to undesriable behavior when using `do`-notation. Thus, we also drop this
`Action` instance and hence force the usage of the `ApplicativeDo` language
extension when using `do`-notation for `Validation`.
CHANGELOG_BEGIN
- [DAML Stdlib] Restrict the ``(>>)`` operator to instances of ``Action``
and make it lazy in its second argument. This gives expressions of the
form ``do A; B`` the desirable semantics of only running ``B`` when
``A`` is a successful action.
- [DAML Stdlib] Remove the ``Action`` and ``ActionFail`` instances for
``Validation`` in ``DA.Validation``. Please enable the ``ApplicativeDo``
language extension if you want to use ``Validation`` with ``do``-notation
and replace ``fail`` with ``DA.Validation.invalid``.
CHANGELOG_END
* Address @cocreature's remarks
CHANGELOG_BEGIN
CHANGELOG_END
We forgot to import the module that defines the duplicate instance
which causes problems if this instance is required for something else,
e.g. because it is a superclass of another instance.
This fixes the issue surfaced by the removal of the Action instance
for Validation in #4552.
changelog_begin
changelog_end
Shouldn’t really make a difference due to laziness but at least it
makes it explicit if we need to decode the archive to the AST or just
need to get the package id.
changelog_begin
changelog_end
On a simple quickstart project we spend about 0.7s in
createProjectPackageDb (all of it in the `withDamlIdeState`
part). This slows down our packaging tests significantly so for now
I’ve added some shortcircuiting if we don’t have any deps and
data-deps. We might want to find a nicer solution here in the
future but the whole code for setting up the package db is going to
move a around a fair bit so I don’t think it makes sense to do this
right now.
changelog_begin
changelog_end
If we get interrupted after updating our local view but before doing
the actual context update on the server side bad things happen. To fix
this, we first clone the context then update the clone which avoids
this and is a bit nicer than throwing an `uninterruptibleMask` around
updating the local view and sending the updateCtx gRPC request.
fixes#4525
changelog_begin
changelog_end
* Fix cross-sdk typeclass reexports in data-dependencies
Sorry for the somewhat large PR, the intermediate states sadly just
change error messages but don’t allow me to test the things they
enable (at least not without considerable effort).
There are a few things this PR fixes:
1. We did not properly expose `dependencies` while compiling
`data-dependencies`. Now they are mapped under a Pkg_$pkgId prefix.
2. We did not pass on `dependencies` to the LF typechecker and
various other places that tried to look up LF packages.
3. We did not filter out duplicate instances, e.g., `daml-prim`
defines `Eq` instances for `Either`. If we define that twice, we
will get an error.
4. The use of Generic in DA.Upgrade doesn’t work
(https://github.com/digital-asset/daml/issues/4470). For now,
I’ve added a somewhat crude hack to filter out typeclasses of a
given name. We should replace this by checking for `Erased`
but that is not yet used in 0.13.51 which I use in the test.
For the test, I’ve checked in a DAR built on 0.13.51 with DAML-LF
1.dev. Not sure if checking in DARs is a reasonable thing to do in the
long term but for now, this is small enough and let’s us move forward
with this.
changelog_begin
changelog_end
* Document envDepInstances
* Document TUnit match
* Update docs for stub binds
* Cleanup and document class rewrites
* Report assistant commands and errors via a logger.
CHANGELOG_BEGIN
CHANGELOG_END
* Hook up daml-assistant to a GCP logger.
* fix test case
* fix more tests
* Check opted in status in assistant.
* Anonimize args that have unexpected characters.
* More agressive anonimization
* add missing containers dependency
changelog_begin
- [DAML Standard Library] Add a ``subtract`` function which is useful
as a replacement for sections of ``(-)``, e.g., ``subtract 1`` is
equivalent to ``\x -> x - 1``.
changelog_end
This is a spin off from my fixes for making data-deps work with
typeclasses cross-SDK.
We only have the interface files for the main dalf so it doesn’t
really make sense to extract the other dalfs. The current behavior of
extracting all dalfs results in them being picked up by
`GeneratePackageMap` even if GHC doesn’t know about them which causes
issues in other placse.
I’ve adapted the collision check to check for transitive dependencies
when creating the project db.
changelog_begin
changelog_end
The string needs to include the full flag instead of only including
--package. Otherwise you get rather unhelpful error messages.
changelog_begin
changelog_end
* Handle multiple constraints on an instance in data-dependencies
Previously we translated `instance (Foo a, Foo b) => Foo (a, b) where`
into `instance Foo a => Foo b => Foo (a, b)` which is a syntax
error. We already did this correctly for `HasField` so this PR mostly
just shuffles things around to always use the non-HasField specific
parts.
changelog_begin
changelog_end
* Prefix fields of DFunHead wtih dfh
* Translate unsupported kinds to a special Erased type
This should simplify `data-dependencies` and avoid issues like #4470
since we can match on the type instead of having to guess which types
can and which cannot be translated back to DAML.
changelog_begin
changelog_end
* Switch to HashSet/HashMap for NormalizedUri/NormalizedFilePath
This matches the change to ghcide in
https://github.com/digital-asset/ghcide/pull/420. Now that we have
optimized Hashable instances it makes sense to use them as much as
possible.
changelog_begin
changelog_end
* debug windows crap
* Fix a bug in vr scenario notifications
* Revert "debug windows crap"
This reverts commit f58fdb92c1.
* attempt to fix windows
who are we kidding, this is not going to work
* Support multiple-packages in `damlc ide`
changelog_begin
- [DAML Studio] You can now open DAML Studio in the root of a
multi-package project instead of opening it separately for each
package. Take a look at the documentation for details on how to set
this up.
changelog_end
There are a few caveats here:
1. You need a ``daml.yaml`` in the root of your project directory. I
think this is somewhat sensible but we should add a warning to VSCode
if you open it in a directory that does not have a ``daml.yaml`` (in a
separate PR).
2. Changes are not picked up accross dependencies. This is a larger
undertaking and given the current setup simply impossible (we don’t
know that the source files of one package belong to the DAR referenced
in the ``dependencies`` field of the other package. We can make this a
bit better by at least detecting that the ``.dar`` has changed but
let’s do that separately.
3. Since ``daml init`` runs once on startup, it will run in the root
directory instead of initializing the package db of the individual
packages. This is fixable but will conflict with #4391 so let’s
address this separately.
I’ve added docs to the daml studio section that explain the caveats.
* Use the proper sdk version in lsp-tests
`fakeDynFlags` has been removed from ghcide. We still need it since we
set up our environment in weird ways and want it to be platform
independent so this PR just copies the definition into `daml`.
changelog_begin
changelog_end
We need to use runActionSync everywhere except for the IDE. Pretty
sure I’ve fixed all cases of those. We also need to use useE instead
of use_ since apparently exceptions shortcircuit shake meaning that
runActionSync doesn’t actually do sync. I’ve only fixed the cases that
we hit in the tests so we probably should do a more thorough pass
after this (I want to get this merged soon since it is quite flaky on
master).
changelog_begin
changelog_end
* Added checks for unit id conflicts.
changelog_begin
changelog_end
* Only report conflicting keys
* lint
* fix indentation
* Show conflicting package ids also
I’ve spend several hours trying to get them to fail on Linux and
Windows and have failed completely (apart from making my machine roun
out of memory, thanks crashplan). I also inserted explicit delays to
make the failure more often frequent without any sucess (at making it
fail). I don’t really have a satisfying explanation of why this is
gone now. My best guess is that the `ghcide` upgrade fixed this since
it included some fixes and changes around how we kill and start the
shake process.
fixes#4328
changelog_begin
changelog_end
Previously, we did not send any message when users simply clicked away
the telemetry popup. Now we send a special message similar to the
opt-out message which only includes the machine id.
I’ve also changed the opt-out message to include the machine id.
changelog_begin
changelog_end
* Use a safer safeToReexport
This is much safer than the approximation from last time. The only downside is
introducing a dependency between data dependencies and our type checker,
but that seems safer than having two versions of `expandTypeSynonyms`
floating around (and perhaps this dependency is something we would end
up adding anyway).
changelog_begin
changelog_end
* Add own package to extPackages.
* Use mkTForalls
* simplify mkTForalls
* Re-export typeclasses in data-dependencies...
... whenever possible.
The `safeToReexport` over-approximation is simple but troubling, so
I'd like to get it fixed soon.
The inputs to generateSrcFromLfPkg (and the genInstances one) were
getting quite hairy, so I decided to pass a record instead.
changelog_begin
changelog_end
* Add a test
* Move generateSrcPkgFromLf to its original place
* Better docs and make envQualify more clear
* compute dependency ids in generate-src command
The packaging tests are already one of our slowest test suites and
damlc test takes quite a while since it has to spin up the scenario
service.
We already have tests for damlc tests so this PR moves the tests from
the packaging test suite that are specific to `damlc test` to those
tests which should balance things a bit better.
changelog_begin
changelog_end
Based on a comment by @cocreature in the last PR, this PR changes the
generation of typeclass instances to empty stubs, instead of adding
stub methods as well.
changelog_begin
changelog_end
Sadly my fix in https://github.com/digital-asset/daml/pull/4325 didn’t
seem to work (despite fixing the issue in my tests) so let’s mark this
as flaky until we figure it out.
changelog_begin
changelog_end
We have seen a couple of flaky tests runs on CI in the packaging test
caused by this. It is easy to reproduce if you add a delay in
diagnosticsLogger before printing the diagnostics which makes this
failure reproducible. With this patch, the diagnostics are always
printed even with a delay.
changelog_begin
changelog_end
* Remove mkOptions and defaultOptionsIO
`mkOptions` is a terrible API and this PR burns it with lots of 🔥.
The only thing that I didn’t simply move around is the validation
logic in `mkOptions` which validates that some directories
exist. Given that I’ve never actually see this be useful and the
options to set these directories (package dbs, import paths) are
internal I don’t think we actually gain anything from this.
I’ve also killed defaultOptionsIO which was a very confusing wrapper
around `mkOptions` that in addition to calling `mkOptions` also
enabled DLint. This was only used in tests, so I’ve enabled DLint in
the tests that need this.
changelog_begin
changelog_end
* newtype version header
* Generate mod refs at the same time as conversion
* Small change
changelog_begin
changelog_end
* lint
* Get rid of secondM
* Switch to Writer.CPS
* doc typo
* Added a partition function to prelude.
* Fixed partition function documentation.
* CHANGELOG_BEGIN
- Added `partition` function to prelude.
CHANGELOG_END
* Fixed partition documentation.
`initPackageDb` assumes that `Options` has not yet been initialized
and calls `mkOptions` itself. Each call to `mkOptions` appends the LF
version to the package db dir which means that calling it twice as we
did in `execIde` results in pkg dbs of the form dir/1.7/1.7 which is
obviously not what we want.
This is just the fix, I’m sufficiently annoyed by this now, that I’ll
spend some time tomorrow to kill mkOptions completely but for now this
at least fixes the SDK on master.
The reason why we didn’t catch this in our tests is that the package
dbs are located slightly differently via Bazel runfiles as they are
located in the final release tarball which ended up not breaking this
in our LSP tests.
changelog_begin
changelog_end
