This PR adds a hook in the compiler and the engine to allow easy prototyping of new features.
In particular by using this hook, one can add a new feature in 1.dev with modifying:
compiler
type checkers (both Haskell and Scala)
archive Protobuf
archive decoder
In addition of development speed, this also adds a bit of confidence, as the peaces enumerated above are not touched when adding the feature: The feature can be added by modifying only 1.dev Daml standard library and Speedy.
aa7991f8 shows a use case of this hook.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
- [LF] Release LF 1.12. This version reduce the size of transaction
- [Compiler]: Change the default LF output from 1.8 to 1.11.
CHANGELOG_END
* Add --target=1.12 support in the compiler.
changelog_begin
- [DAML Compiler] Add support for ``--target=1.12`` in the DAML compiler.
changelog_end
* version1_12 not version1_11
* Update codegen tests.
* Update codegen tests again
* Fix data-dependencies test.
* Update protobuf to throw exceptions directly.
Part of #8020. This PR changes the exception protobuf and AST (Haskell
side) to make exceptions be thrown directly via a primitive expression
(EThrow), instead of wrapping them up via AnyException.
changelog_begin
changelog_end
* Rename MakeAnyException to ToAnyException
* update EToAnyException field names
* Missing stuff
* missing scala case
* Make AnyException unserializable
* reindex protobuf builtins
* meaningless change
* refrobulate
* change pretty
* Add DAML_EXCEPTIONS feature.
Part of #8020. This extracts the DAML_EXCEPTIONS feature from #8023
since we need it for typechecking & LF conversion soon.
changelog_begin
changelog_end
* add full url
* add another url
* Add exceptions in DAML-LF AST (Haskell side)
Updates the Haskell DAML-LF AST for exceptions, including
encode/decode, and updates all the functions that deal with the
AST directly.
changelog_begin
changelog_end
* Missed some TODOs in DecodeV1
Some features, in particular daml-lf encoding features, don't make any sense to have an associated CPP flag. I made the CPP flag optional and removed these unnecessary feature flags (based mostly on whether the flags were used in the standard library).
changelog_begin
changelog_end
* DAML-LF: Add interning for type to DAML-LF 1.dev
We add two new features to DAML-LF 1.dev:
* a per package list (or table) of `Type` messages, and
* a new case in the `Type` message which is an index into this table.
In combination, these two features can be used to allow DAML-LF
encoders to perform hash-consing of `Type` messages. We also change the
Haskell implementation of our DAML-LF encoder to do exactly that when
targetting DAML-LF 1.dev.
Doing this has a few benefits:
1. The DALFs produced by `damlc` get smaller: I've seen a case where
the size dropped from 69MB to 45MB.
2. DAML-LF decoders need to decode less data.
3. Decoded packages use less memory because identical structures are
now shared. This is particularly helpful in situations where we need
to keep the interface (or signature) of a package in memory for a
long time.
This PR mostly takes care of the Haskell implementation. However, we
need to make the Scala implementation of the decoder aware of the new
features as well since we have tests that load DAML-LF 1.dev into the
engine. A decoder and _targeted_ tests on the Scala side will follow
in a separate PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Make jq tests aware of type interning
CHANGELOG_BEGIN
CHANGELOG_END
* Improve jq test
CHANGELOG_BEGIN
CHANGELOG_END
* Apply Remy's suggestions
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Improve the imperative bits
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Haskell: Add hlint rule to suggest foldl' over foldl
`foldl` is lazy in a way that almost never is what you want since it
can cause space leaks without any benefit. `foldl'` does not have this
problem. See https://www.well-typed.com/blog/2014/04/fixing-foldl/ for
more details.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix all existing occurrences of foldl
CHANGELOG_BEGIN
CHANGELOG_END
Currently, the Haskell implementation of the DAML-LF type checker
first branches on the pattern and then on the type of the scrutinee.
This PR changes it to first branch on the type of the scrutinee and
then on the pattern. This allows for destructing the type of the
scrutinee only once intead of repeating it for each pattern. This
should be good for performance, which is a nice side effect of this
change.
The main reason why I'm changing this is because we want to implement
an exhaustiveness check. This seems rather complicated to achieve with
the current implementation and will be significantly easier after this
change.
This PR is purely a refactoring and does not change the semantics of
amnything. In particular, it does not touch the decision that default
patterns match on anything, even no other pattern would match on the
value.
CHANGELOG_BEGIN
CHANGELOG_END
* choice observers, WIP
changelog_begin
changelog_end
fix generator driven test
Node.isReplayedBy, consider observers
add observers to NodeExercise in transaction.proto
add observers to TemplateChoice in Ast.scala
add observers to LF .proto, and Haskell Ast for TemplateChoice
reinstate trailing // for better format
fix validate tests
fix haskell LF decoder when choice-observers field is missing in .proto
fix build
make choice-observers optional in scala AST
make choice-observers optional in Haskell Ast
address comments from Remy and Martin
more review comments
check TransactionVersions.minChoiceObservers in Transaction encode/decode
featureChoiceObservers, and check in haskell type-checker
improve speedy Compiler for empty choice-observers
extend scala LF decoder for optional choice observers
extend scala parser for choices to allow optional choice-observers clause, and test
rename new field in scala Ast -> "choiceObservers"
var rename
extend TypingSpec tests for choice-observers. also add missing negative test for controllers
switch from keyword "ob" to identifier "observers" in scala parser choice syntax
add TODO for featureChoiceObservers to be part of DAML 1.9 (issue 7139)
* replace "NICK" comment markers with "FIXME #7709" comment markers
This PR adds default method support in data-dependencies,
including default method signatures (the DefaultSignatures
extension). This adds tests for simple default methods,
and for default methods with signatures.
changelog_begin
- [DAML Compiler] `data-dependencies` now support default
methods in typeclasses, including default method
type signatures.
changelog_end
An extraction script that makes the LF reading libraries (DAML LF archive decoding to an AST) usable in other Haskell projects.
This script once existed but is not available any more, so I have resurrected the script, adapted to current usage, and upgraded to the newest stackage LTS that can support it. These libraries _could_ be versioned manually to released SDKs and published if desirable (without making any guarantees about the stability, therefore prefixing the version with `0.`.
CHANGELOG_BEGIN
Tool to extract Haskell libraries for DAML LF archives for use in other Haskell projects
CHANGELOG_END
fixes#7114
This PR changes the Show instance of ContractId and flips the switch
on triggers and DAML Script to run in off-ledger mode.
It also adds a test that for DAML Script we actually get back the
correct contract id.
There is a bit of a design decision here in how we want to print
contract ids, so let me list the options I considered. $cid will stand
for the actual cid and all options are wrapped in markdown inline
code.
1. `"$cid"`. Indistinguishable from string. Suggests that there might
be an IsString instance for ContractId.
2. `<$cid>`. Matches the dummy `<contract-id>` but it’s not a dummy so
I don’t think matching that is benefitial.
3. `$cid`. Easy to spot (contract ids start with # and have no
spaces), clearly not a string but might look slightly weird.
changelog_begin
- [DAML Script/DAML Triggers] When using DAML-LF 1.dev, the `Show` instance of `ContractId` will now display the actual contract id instead of a dummy `<contract-id>` value. Note that this only applies to DAML Script and DAML Triggers not to ledger code.
changelog_end
* Add TO_TEXT_CONTRACT_ID primitive
This is the first part of #7114.
This PR
* Adds the primitive to the protobuf.
* Handles decoding and encoding in Haskell and Scala.
* Handles typechecking in Haskell and Scala.
* Handles speedy compilation and interpretation in Scala.
* Updates the specification.
This PR does not yet change the standard library to make use of this
primitive.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Avoid extra allocation
changelog_begin
changelog_end
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* First draft of constant lifting
changelog_begin
changelog_end
* refactoring
* doing stuff
* run simplifier on template exprs
* remove merge artifact
* Fix ExerciseWithoutActors
* add comments
* fix trace order test
* prefix generated val names with their provenance
* Verification tool bugfix
During the value collection phase, when encountering a record projection on a (yet) undefined value, stop searching this branch instead of throwing an error.
* Bump pattern-match-perf memory limit with cocreature`s blessing
* bump again
* Filter generated identifiers from daml script test runner
changelog_begin
changelog_end
* Fix party literals
* Remove inlineClosedExpr for now.
* Improve comments
* Reset script test locations
* Unhashmap
* disable daml-lf-verify quickstart tests for now
Co-authored-by: Gert-Jan Bottu <gertjan.bottu@kuleuven.be>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* damlc inspect: Add output modes with less details
This PR adds two new detail levels to `damlc inspect`:
* `--detail=-1` omits all package ids from the pretty printed output.
* `--detail=-2` omits all package ids, type information and kind
annotations from the output.
We also slightly change the existing detail level `--detail=0`, which
is the default when the `--detail` flag is omitted, to make the output
more consistent:
* Kind annotations are no longer omitted when the kind is `*` (star).
* All location information is now omitted. To get location information,
use `--detail=1`.
This is part of https://github.com/digital-asset/daml/issues/5756.
CHANGELOG_BEGIN
CHANGELOG_END
* Improve explanatory comment
CHANGELOG_BEGIN
CHANGELOG_END
Clean up the DAML-LF pretty printer used by `damlc inspect` (and
others).
Most of the changes are around using `pPrintPrec` or `pPrint` instead
of `pretty` and fixing some naming inconsistencies. In many places we
now use `pPrintPrec` because I'd like to support multiple levels of
details in a follow-up PR. This requires passing the `PrettyLevel` down
the AST.
We also remove 4 leading spaces from almost every line since they cost
precious screen space and don't add any value.
This is part of https://github.com/digital-asset/daml/issues/5756.
CHANGELOG_BEGIN
CHANGELOG_END
* Add a module-prefixes field to rename all modules in a pkg
This PR adds a `module-prefixes` field to `daml.yaml` a a shorthand
for specifying a `--package` flag that renames all modules in a
package to have the same prefix.
The docs are updated to describe how you can use this field and there
is a test case that makes sure it works.
fixes#4948
changelog_begin
- [DAML Compiler] You can now use the new ``module-prefixes`` field in
``daml.yaml`` to add a prefix to all modules from a dependency. This
is particularly useful for handling colliding module names during
upgrades. See
https://docs.daml.com/daml/reference/packages.html#handling-module-name-collisions
for more information.
changelog_end
* Update compiler/damlc/daml-opts/daml-opts-types/DA/Daml/Options/Packaging/Metadata.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Improve docs and fix broken suggestion
changelog_begin
changelog_end
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Add JSON output to `damlc inspect-dar`
fixes#3117, fixes#5563
This PR adds a `--json` flag to `damlc inspect-dar` that does pretty
much what you expect. So far it mainly exposes the same information
that you already got from `inspect-dar` but it’s easy to add more as
needed.
I’ve also finally added some documentation for `inspect-dar`.
changelog_begin
- [DAML Compiler] ``damlc inspect-dar`` now has a `--json`` flag to
produce machine-readable output. See
https://docs.daml.com/daml/reference/packages.html#inspecting-dars
for more information.
changelog_end
* Sort packages in human-readable output
changelog_begin
changelog_end
* Update docs/source/daml/reference/packages.rst
Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
First version of static verification tool.
The current state of the tool:
- Reads DAR files.
- Partially evaluates the code.
- Generates constraints for the field and choice to be verified.
- Passes the constraints to an SMT solver.
- Some basic tests.
* Inline typeclasses and apply projections.
This draft PR:
* adds a Subst module for substitution within LF expressions.
This implementation piggybacks on the existing type
substition. But there are not enough tests yet.
* passes World data into the simplifier, in order to have
have the ability to inline functions (selectively)
* inlines typeclass dictionaries and projection functions
* beta reduces type lambdas (which are just ignored by
speedy afaik)
* beta reduces lambdas that are passed units or dictionaries
* runs the simplifier twice so it has a change to
perform the inlining, reduction, AND projection
-- this is probably avoidable by sequencing the
simplifier steps in a specific order, but running
the simplifier twice is simple enough for now.
* together, these fix#5748 (see result on a toy module)
TODO:
* add lots of tests for Subst
* run this on a benchmark to see how big of a difference it makes.
* reduce the jankiness of running the simplifier twice.
Results:
DAML input:
```
module Main where
hello : Int
hello = 10 + 30
```
Original DAML-LF output:
```
module Main where
@location(7:0-7:5)
def hello : Int64 =
a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:+
@Int64
a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:$fAdditiveInt
10
30
```
Current DAML-LF output:
```
module Main where
@location(7:0-7:5)
def hello : Int64 = ADD_INT64 10 30
```
changelog_begin
changelog_end
* Update copyright header
* Lint
* More direct typeclass simplification
* extendWorldSelf + getTypeClassDictionary
* Fix visual
* Fix visual comment
* Disable cross-module inlining for incremental builds
* Cleanup subst/freevars.
* copyright header
* Alpha equivalence for LF expressions.
* copyright header
* lots of tests
* fix comment
* Apply review suggestions
* Fix handling of packages in damlc visual
Previously we just ran the analysis on the modules of the main
package. This failed for obvious reasons as soon as you reference a
template from another package which happens pretty
frequently (e.g. for anything that uses finlib).
This PR fixes this to run the analysis on the whole World which is
self-contained. This required a bunch of reshuffling to make sure that
we always reference fully qualified identifiers but most of it is
very mechanical.
Note that currently you cannot distinguish between templates with
identical names in the resulting graph (they will be separate but you
have no idea which one is which). This was already an issue
before if you have the same template name in different modules so I
consider this an orthogonal issue.
This fixes the expected failure we already had and I added another
test that checks that colliding template names do at least show up as
separate nodes in the graph. I also manually tested this against
ex-bond-issuance.
Disclaimier: I’m aware that the code is very messy but I tried to
resist the urge to rewrite it completely and only change what was
necessary.
fixes#5776
changelog_begin
- [DAML Compiler] ``damlc visual`` now works properly in projects
consisting of multiple packages.
changelog_end
* Rename templateChoiceId to templateId
changelog_begin
changelog_end
DAML-LF 1.6 is the baseline vesion we want to support with the next
SDK release.
CHANGELOG_BEGIN
- [DAML Compiler] Drop support for reading DARs using DAML-LF 1.5.
Please recompile your DAML to a newer DAML-LF version in case you
have such DARs.
CHANGELOG_END
* Make name collision check more strict
This PR extends the name collision check to catch collisions between
A:B (type B in module A) and module A.B.C. For now this is just a
warning and not an error. Once we turn it into an error, we also need
to add this to the Scala collision checker.
There is a fair bit of plumbing required to make warnings work but on
the plus side we get multiple errors at once now instead of erroring
out on the first one.
changelog_begin
- [DAML Compiler] The name collision check has been extended to also
count the case as a collision where you have a type B in module A and a module
A.B.C (but no module A.B). This is a warning in this SDK release but
will become an error in a future release. The typescript codegen is
not usable on packages that don’t uphold this restriction.
changelog_end
* Address review comments
changelog_begin
- [DAML Compiler] The default output DAML-LF target version is now
1.8. You can target 1.7 by specifying ``--target=1.7`` in the
``build-options`` field in your ``daml.yaml``.
changelog_end
Fix daml2ts tests
This PR exposes the new generic comparison primitives, preferring them
to the older comparison primitives whenever they are available. It also
separates the generic comparison feature from the GenMap feature,
because that just makes sense.
changelog_begin
changelog_end
* Freeze DAML-LF 1.8
Two minor points that I did not mention in the previous PR:
We also include the renaming of structural records to `struct` and the
renaming of `Map` to `TextMap`.
There are some minor changes around the LF encoder tests which need to
be able to emit package metadata properly so I’ve added it to the
parser. Sorry for not splitting that out.
Following the process used for the DAML-LF 1.7 release, this does not
yet include the frozen proto file.
changelog_begin
- [DAML-LF] Release DAML-LF 1.8:
* Rename structural records to ``Struct``. Note that
structural records are not exposed in DAML.
* Rename ``Map`` to ``TextMap``.
* Add type synonyms. Note that type synonyms are not serializable.
* Add package metadata, i.e., package names and versions.
Note that the default output of ``damlc`` is stil DAML-LF 1.7. You
can produce DAML-LF 1.8 by passing ``--target=1.8``.
changelog_end
* Update encoder
* Update java codegen tests
* Update comment in scala codegen
* Handle TSynApp in interface reader
* Bump lf_stable_version to 1.7
* Fix kvutils tests
We will only include type synonyms and package metadata (which are
used for Cross-SDK model upgrades). Everything else stays in 1.dev for
now.
changelog_begin
changelog_end
* Use package metadata instead of file names to infer unit ids
This PR adds a function that abstracts over whether we get metadata
from a filename (< 1.dev) or directly from the LF metadata.
There is more work to be done here, in particular, I want to clean up
the hacks around daml-prim/daml-stdlib but I’ll leave that for a
separate PR.
changelog_begin
changelog_end
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Util.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
* Refactor getUnitId
Co-authored-by: associahedron <231829+associahedron@users.noreply.github.com>
This PR adds the necessary infrastructure to produce package metadata
in `damlc`.
For `damlc build` this works exactly as expected. There are a few edge
cases where we don’t have names and/or versions (namely scenarios,
damlc compile and damlc package). We don’t really care about the
metadata for those anyway, so I’ve just set it some default value.
changelog_begin
changelog_end
* Refactor handling of package names and versions
This is a preparatory refactoring PR in preparation for propagating
package metadata into DAML-LF. There are no actual changes in here.
Primarily the changes consist of 3 things:
1. In options, we split the `optMbPackageName` field which previously
contained the unit id into `optMbPackageName` and
`optMbPackageVersion`.
2. We use newtypes for names and versions and try to keep them pretty
much everywhere (the only place missing is `splitUnitId`, I’ll do that
separately).
3. We use `UnitId` where we want `name-version`.
As was probably to be expected, this surfaced some minor issues. They
are pretty much exclusively in debugging or “internal” commands so
I’ve mostly just added notes/todos.
changelog_begin
changelog_end
* cry about applicativedo
* Add package metadata to DAML-LF proto and the Haskell AST
This adds package metadata (currently only the package name and
version) to DAML-LF and the corresponding Haskell ASTs. This is useful
for debugging and “codegens” (typescript, damlc dependencies, …)
This PR does not yet add it to the Scala side or change the compiler
to actually produce this metadata.
Part of #4412
changelog_begin
changelog_end
* Address review comments
* Use a safer safeToReexport
This is much safer than the approximation from last time. The only downside is
introducing a dependency between data dependencies and our type checker,
but that seems safer than having two versions of `expandTypeSynonyms`
floating around (and perhaps this dependency is something we would end
up adding anyway).
changelog_begin
changelog_end
* Add own package to extPackages.
* Use mkTForalls
* simplify mkTForalls
