* First draft of constant lifting
changelog_begin
changelog_end
* refactoring
* doing stuff
* run simplifier on template exprs
* remove merge artifact
* Fix ExerciseWithoutActors
* add comments
* fix trace order test
* prefix generated val names with their provenance
* Verification tool bugfix
During the value collection phase, when encountering a record projection on a (yet) undefined value, stop searching this branch instead of throwing an error.
* Bump pattern-match-perf memory limit with cocreature`s blessing
* bump again
* Filter generated identifiers from daml script test runner
changelog_begin
changelog_end
* Fix party literals
* Remove inlineClosedExpr for now.
* Improve comments
* Reset script test locations
* Unhashmap
* disable daml-lf-verify quickstart tests for now
Co-authored-by: Gert-Jan Bottu <gertjan.bottu@kuleuven.be>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This integrates the time service into DAML script thereby covering the
main piece of scenarios that was missing from DAML script.
This PR does two things (they are very related and doing them together
makes it much easier to test):
1. It “fixes” `getTime` to return the ledger time in static mode by
querying the ledger time service instead of defaulting to the Unix
epoch which is pretty useless and I would consider the old behavior
a bug. We keep the old behavior via the JSON API since there is no
time service.
2. It adds `setTime` to set the ledger time via the time service. This
is only supported in static time mode (sadbonx and other ledgers do
not expose the time service in wallclock mode because changing time
makes it not wallclock) or via the JSON API (no time service).
fixes#6220
changelog_begin
- [DAML Script] DAML Script’s `getTime` now correctly handles time
changes in static time mode and returns the current time by querying
the time service rather than defaulting to the Unix epoch. Note that
when run via the JSON API, it still returns the Unix epoch.
- [DAML Script] Add `setTime` to DAML Script which sets the ledger
time via the ledger API time service. Note that this is only
supported when running over gRPC in static time mode.
changelog_end
In this PR we cleanup the constructor for the speedy Machine.
* We remove the `case` keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
+ one generic builder `Machine.apply`,
+ scenario specific builder,
CHANGELOG_BEGIN
CHANGELOG_END
The upgrade to various nodejs libraries in
b03cf7b598
seems to have gotten in an inconsistent state (what even are version
boundaries). This PR updates vscode-languageclient to the latest
version which fixes the issue. Apparently that requires a newer
version of @types/vsode as well (even 6.0.x requires that) so I had to
bump that and the engine. This does mean that we now require vscode
1.39.0. Given that this version was released in September 2019 that
seems very reasonable.
changelog_begin
- [DAML Studio] DAML Studio now requires VSCode 1.39 or newer.
changelog_end
fixes#3150
This PR introduces a patch to GHC to fix the performance of the
pattern match checker in the presence of multiple packages which
is currently significantly (orders of magnitude) slower than having
everything in a single package. I also added a test case that hits
this. Here’s what you need to hit this issue:
1. A typeclass with a functional dependency. `HasField` is the obvious
candidate for this.
2. A lot of instances of this typeclass in a separate package (this is
the only part where the separate package matters).
3. A reasonably large ADT with a bunch of strict fields.
4. A pattern match in the context of some constraints of the
typeclass. The constraints can be completely unused.
In that case, you will get a significant slowdown in the number of
instances, number of constructors and number of constraints (didn’t
verify if it’s linear but it is significant which is all that
matters).
Here’s why this happens:
1. The pattern match checker checks for strict fields if the type is
inhabited.
2. This calls `pmTopNormaliseType_maybe` to normalize a type (the details don’t
matter) which in turn calls into the typechecker. This function is
called very often (presumably linear in the number of constructors
but didn’t verify.)
3. The typechecker has some logic in `improveFromInstEnv` for
generating additional equations by unifying functional
dependencies `a -> b` with constraints in scope
and thereby deducing information about `b`.
4. In the pattern match checker the list of instances of the home
package is empty since the pattern match checker (apparently)
doesn’t actually care about those extra equations. However, the
list of instances in the EPS is not empty. This is the issue here:
By moving it to an external package we suddenly end up with
thousands of instances that we try to unify with the functional
dependencies every time we normalize which happens very often.
Proposed fix:
The solution is rather simple: Since the pattern match checker
apparently does not care about the instances of the home package, it
almost certainly doesn’t care about instances in general so we just
empty the instances of external packages explicitly.
Is the fix correct?
1. I verified that the GHC test suite passes with this patch which
gives me a reasonable level of confidence.
2. I verified that our own test suite passes.
3. The most dodgy part is actually emptying the instance since the
whole EPS stuff is a mutable mess. What could in theory happen is that
the PM ends up loading an interface file that mutates this
again. However, afaiu it is impossible for the PM to need an
interface that the typechecker didnt already need. I did do a bunch
of debugging and this is exactly what I observed in my experiments.
Alternative ideas and upstreaming:
The other option would be to not try and mess with the EPS but somehow
have a conditional flag somewhere in the typechecker env to disable
this logic in the pattern match checker. However, that sounds
significantly more complex so I don’t think it’s worth the effort.
GHC 8.10 has a new pattern match checker that has different
performance characteristics and seems to do much better here so there
is little reason to try and upstream this. I strongly want to avoid
upgrading DAML to 8.10 at this point (too much risk, let’s wait until
things calm down)
changelog_begin
- [DAML Compiler] Fix an issue where compilation slowed down
significantly when code was split up into several packages. See
https://github.com/digital-asset/daml/issues/3150
changelog_end
Doctests generate a temporary file in .daml/generated. This works fine
as long as you only run one instance at the same time. However, those
tests here run in parallel and they run in the same working directory
even though we create a temporary directory for the source file. This
means that occasionally (very rarely, I still haven’t managed to
reproduce it locally) the two tests overwrote each other’s file and
you end up with a mess.
There are two solutions here:
1. Disable parallel tests.
2. Remove the working directory as global state and instead use tmpDir
as the working directory.
Nobody likes global state so I went with 2.
I ran 500 iterations of this test on CI on all 3 platforms and got not
a single failure so I’m reasonably confident this fixes the issue.
changelog_begin
changelog_end
* damlc: Run simplifier on templates as well
Currently, the simplifier only runs on top-level value definitions.
However, there's no good reason for not running it on all expressions
within templates as well.
This does not significantly improve the `CollectAuthority` benchmark,
but there's not much code in the templates.
CHANGELOG_BEGIN
CHANGELOG_END
* Verification tool bugfix in variable projection and substitution with… (#6344)
* Verification tool bugfix in variable projection and substitution within let expressions
* Remove redundant substitution ; apply feedback Martin
Co-authored-by: Gert-Jan Bottu <gertjanbottu@hotmail.com>
* update various lock files
CHANGELOG_BEGIN
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* further version upticks from yarn upgrade
* Add option based constructor for LedgerIdRequirement
changelog_begin
changelog_end
* Make option based consructor the default, deprecate old constructor
* Update with review comments
* damlc inspect: Add output modes with less details
This PR adds two new detail levels to `damlc inspect`:
* `--detail=-1` omits all package ids from the pretty printed output.
* `--detail=-2` omits all package ids, type information and kind
annotations from the output.
We also slightly change the existing detail level `--detail=0`, which
is the default when the `--detail` flag is omitted, to make the output
more consistent:
* Kind annotations are no longer omitted when the kind is `*` (star).
* All location information is now omitted. To get location information,
use `--detail=1`.
This is part of https://github.com/digital-asset/daml/issues/5756.
CHANGELOG_BEGIN
CHANGELOG_END
* Improve explanatory comment
CHANGELOG_BEGIN
CHANGELOG_END
* damlc: Avoid immediately invoked closures
We add rules to the simplifier which rewrite expressions of the form
```
(\x1 ... xn -> E) A1 ... An
```
into
```
let x1 = A1 in ... let xn = An in E
```
provided that `xi` is not free in `Aj` for any `i < j`.
In cases where `E` is _not_ a lambda, this rewriting is beneficial
since it removes a costly closure allocation and the immediate entering
of the closure. If `E` is a closure itself, this rewriting is not
detrimental since the only difference it makes is that the values for
`A1, ..., An` end up in the list of captured variables rather than in
the list of already applied variables.
Even though users might not write expressions like the one we're
simplifying here themselves, various desugarings produce them
nevertheless. A common pattern is to define multiple auxiliary
functions that a only used once in a where cluase. For instance, with
the help these new rewriting rules the function
```haskell
f: Int -> Int -> Int
f x y = g (h x) y
where
g x y = x+y
h x = 2*x
```
gets translated to
```
def f : Int64 -> Int64 -> Int64 =
\(x : Int64) (y : Int64).
let x2 : Int64 =
let x2 : Int64 = x
in MUL_INT64 2 x2
y2 : Int64 = y
in ADD_INT64 x2 y2
```
Without the simplification, `g` and `h` would each allocate a closure:
```
def f : Int64 -> Int64 -> Int64 =
\(x : Int64) (y : Int64).
(\(x2 : Int64) (y2 : Int64). ADD_INT64 x2 y2)
((\(x2 : Int64). MUL_INT64 2 x2) x)
y
```
The `collect-authority` benchmarck is sped up by 1.03x by this change.
This is not a huge improvement but the change is simple enough to
merge it nevertheless.
CHANGELOG_BEGIN
CHANGELOG_END
* Let code speak not comments
CHANGELOG_BEGIN
CHANGELOG_END
* Improve explanations
CHANGELOG_BEGIN
CHANGELOG_END
Clean up the DAML-LF pretty printer used by `damlc inspect` (and
others).
Most of the changes are around using `pPrintPrec` or `pPrint` instead
of `pretty` and fixing some naming inconsistencies. In many places we
now use `pPrintPrec` because I'd like to support multiple levels of
details in a follow-up PR. This requires passing the `PrettyLevel` down
the AST.
We also remove 4 leading spaces from almost every line since they cost
precious screen space and don't add any value.
This is part of https://github.com/digital-asset/daml/issues/5756.
CHANGELOG_BEGIN
CHANGELOG_END
We split the object com.daml.lf.types.ledger in three:
- one part in `com.daml.lf.ledger.` (in //daml-lf/transaction) for the part relative to EventId (shares between scenario service and sandbox)
- one part in `com.daml.lf.ledger.` (in //daml-lf/interpreter) for the part common to Blinding and Scenario
- one part in `com.daml.lf.scenario.` (in //daml-lf/interpreter) for the part specific to Scenario
fixes#6260
CHANGELOG_BEGIN
CHANGELOG_END
This is just not general enough to be useful. In particular, it
completely falls apart for somethin like submit x (exercise …) in
scenarios.
changelog_begin
changelog_end
This rule is repeated for every file. While we cache the computation,
we still forced it to NF which is super slow. On my (realworld)
testcase, this is a speedup of > 1.7x, cuts allocations to 1/3 and max
residency also goes down to 1/3.
changelog_begin
changelog_end
* replace NodeExercises#controllers with controllersDifferFromActors
* remove controllers from ActorMismatch and scenario service exercise
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove reserved ID #s in scenario-service grpc
As discussed, we don't need to worry about
version mismatches.
Co-authored-by: Remy <remy.haemmerle@daml.com>
Previously, we just crashed the scenario service instead of throwing a
proper scenario error. This meant that you had to look at the
debugging output to figure out what is going wrong. This is both a
shitty UX and also inconsistent with how we handle this for fetch and
exercise on contract ids that are not visible. This PR adds a new
error type that matches the one for invisible contract ids.
changelog_begin
- [DAML Studio] Fetches and exercises of contract keys associated with
contracts not visible to the submitter are now handled properly
instead of showing a low-level error.
changelog_end
fixes#5903
* Add a module-prefixes field to rename all modules in a pkg
This PR adds a `module-prefixes` field to `daml.yaml` a a shorthand
for specifying a `--package` flag that renames all modules in a
package to have the same prefix.
The docs are updated to describe how you can use this field and there
is a test case that makes sure it works.
fixes#4948
changelog_begin
- [DAML Compiler] You can now use the new ``module-prefixes`` field in
``daml.yaml`` to add a prefix to all modules from a dependency. This
is particularly useful for handling colliding module names during
upgrades. See
https://docs.daml.com/daml/reference/packages.html#handling-module-name-collisions
for more information.
changelog_end
* Update compiler/damlc/daml-opts/daml-opts-types/DA/Daml/Options/Packaging/Metadata.hs
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Improve docs and fix broken suggestion
changelog_begin
changelog_end
Co-authored-by: Martin Huschenbett <martin.huschenbett@posteo.me>
* Add JSON output to `damlc inspect-dar`
fixes#3117, fixes#5563
This PR adds a `--json` flag to `damlc inspect-dar` that does pretty
much what you expect. So far it mainly exposes the same information
that you already got from `inspect-dar` but it’s easy to add more as
needed.
I’ve also finally added some documentation for `inspect-dar`.
changelog_begin
- [DAML Compiler] ``damlc inspect-dar`` now has a `--json`` flag to
produce machine-readable output. See
https://docs.daml.com/daml/reference/packages.html#inspecting-dars
for more information.
changelog_end
* Sort packages in human-readable output
changelog_begin
changelog_end
* Update docs/source/daml/reference/packages.rst
Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
Co-authored-by: Rohan Jacob-Rao <rohanjr@gmail.com>
Choices for `stacktracing` are `NoStackTrace` / `FullStackTrace`.
Adapt code so the selection is made by the original caller:
- `engine`
- `scenario-service`
- `repl-service`
- `daml-script` runner
etc
Currently, all callers pass `FullStackTrace` (the existing behaviour), except for the
exploration dev-code: `daml-lf/interpreter/perf/src/main/scala/com/daml/lf/explore`.
The idea is that once this control is in place, we can discuss if we can change how we
might expose it to the user, and/or perhaps change the default behaviour to have
`stacktracing` disabled.
changelog_begin
changelog_end
First version of static verification tool.
The current state of the tool:
- Reads DAR files.
- Partially evaluates the code.
- Generates constraints for the field and choice to be verified.
- Passes the constraints to an SMT solver.
- Some basic tests.
* REPL record dot syntax regression test
* REPL: Apply record preprocessor to expressions
CHANGELOG_BEGIN
- [DAML REPL] Record dot syntax is now handled in expressions entered
into the REPL.
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Implement a simple profiler for DAML scenarios
The profiler runs a single scenario and records timing information when
each function (and some other closures) are entered and left. The
resulting information can be visualized as a flamegraph using
[speedscope](https://www.speedscope.app/).
The profiler works by instrumenting the CEK machine at the heart of
DAML Engine. Unfortunetaly, this causes a very small overhead on
non-profiling runs too. However, in my benchmarks I could not measure
any significant impact on the overall runtime at all. More precisely,
the overhead is as follows:
Every closure now has an additional field called `label`. In
non-profiling runs this field is always set to `null`. This field needs
to be allocated, copied whenever we copy a closure and scanned during
garbage collection. Additionally, whenever we enter a closure, we check
this field and whenever it is _not_ `null`, i.e. never during
non-profiling runs, we record an "open event" and set up a hook for the
corresponding "close event". Thus, the additional cost during
non-profiling runs are a single pointer comparison and a jump beyond
the "then branch".
Since this is still very much in active development, there are no
documentation, other than an entry in a README, and no tests yet. They
will come before we promote this. However, the UX will look very
different then since we already have plans to significantly change it.
CHANGELOG_BEGIN
CHANGELOG_END
* Run scalafmt
* Make profiling argument to PureCompiledPackges optional
* Fix a bunch of tests
CHANGELOG_BEGIN
CHANGELOG_END
* scalafmt is so annoying
CHANGELOG_BEGIN
CHANGELOG_END
* Apply simple suggestions
CHANGELOG_BEGIN
CHANGELOG_END
fixes#6005
Take a look at #6005 for details. I verified manually that things
still work as expected. It does require bumping the vscode version to
1.38 which was released almost a year ago so hopefully that should be
safe.
changelog_begin
changelog_end
Fixes#5592
The CLI syntax and the defaults follow the JSON API here.
changelog_begin
- [DAML Script] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON
API.
- [DAML REPL] The maximum inbound message size can now be configured
using `--max-inbound-message-size``. This matches the flag in the JSON API.
changelog_end
changelog_begin
dlint: Change references to replicateM to replicateA; remove some Haskell-based rules that do not currently apply in DAML.
changelog_end
* Inline typeclasses and apply projections.
This draft PR:
* adds a Subst module for substitution within LF expressions.
This implementation piggybacks on the existing type
substition. But there are not enough tests yet.
* passes World data into the simplifier, in order to have
have the ability to inline functions (selectively)
* inlines typeclass dictionaries and projection functions
* beta reduces type lambdas (which are just ignored by
speedy afaik)
* beta reduces lambdas that are passed units or dictionaries
* runs the simplifier twice so it has a change to
perform the inlining, reduction, AND projection
-- this is probably avoidable by sequencing the
simplifier steps in a specific order, but running
the simplifier twice is simple enough for now.
* together, these fix#5748 (see result on a toy module)
TODO:
* add lots of tests for Subst
* run this on a benchmark to see how big of a difference it makes.
* reduce the jankiness of running the simplifier twice.
Results:
DAML input:
```
module Main where
hello : Int
hello = 10 + 30
```
Original DAML-LF output:
```
module Main where
@location(7:0-7:5)
def hello : Int64 =
a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:+
@Int64
a284919a95c4a515cd1efac0d89be302d0e9d61e692a2176128c871ad8067e36:GHC.Num:$fAdditiveInt
10
30
```
Current DAML-LF output:
```
module Main where
@location(7:0-7:5)
def hello : Int64 = ADD_INT64 10 30
```
changelog_begin
changelog_end
* Update copyright header
* Lint
* More direct typeclass simplification
* extendWorldSelf + getTypeClassDictionary
* Fix visual
* Fix visual comment
* Disable cross-module inlining for incremental builds
* Cleanup subst/freevars.
* copyright header
* Alpha equivalence for LF expressions.
* copyright header
* lots of tests
* fix comment
* Apply review suggestions
Fixes#5786.
CHANGELOG_BEGIN
[DAML Standard Lib] ``DA.Text.splitOn`` will now correctly handle the
case where the separator appears at the end but should not be matched,
as in ``splitOn "aa" "aaa" == ["", "a"]`` (previously this was
erroneously returning ``["", "a", ""]``). See #5786 for more details.
CHANGELOG_END
* Simplify and clarify the public interface to Speedy.
- Remove `isFinal`. A client just uses `run()`.
- Remove `toSValue`. The value in available in `SResultFinalValue(v: SValue)`.
- A client never directly access the `.ctrl` (or `.returnValue`) components.
- A client may use `setExpressionToEvaluate(expr)` to evaluate a new expression on an existing machine.
changelog_begin
changelog_end
* remove while loop which executes just once
* avoid unnecessary mutation when running speedy
--project-root is a bit confusing when using the assistant since it
will choose the SDK version before going to the project
directory. This is almost never what you intend to do so setting
DAML_PROJECT seems like a better option in most cases.
See https://github.com/digital-asset/daml/issues/5769 for details
changelog_begin
changelog_end
* CHANGELOG_BEGIN
Added filterA to Prelude.
* Update compiler/damlc/daml-stdlib-src/DA/Internal/Prelude.daml
Co-authored-by: Shayne Fletcher <shayne@shaynefletcher.org>
* Removed not so useful comment.
* Moved filterA from Prelude to Action.
* filterA is a one-liner now.
* Provided more meaningful example to filterA.
* Added test for filterA.
* Removed failing doctest.
CHANGELOG_END
Co-authored-by: Shayne Fletcher <shayne@shaynefletcher.org>
* Fix handling of packages in damlc visual
Previously we just ran the analysis on the modules of the main
package. This failed for obvious reasons as soon as you reference a
template from another package which happens pretty
frequently (e.g. for anything that uses finlib).
This PR fixes this to run the analysis on the whole World which is
self-contained. This required a bunch of reshuffling to make sure that
we always reference fully qualified identifiers but most of it is
very mechanical.
Note that currently you cannot distinguish between templates with
identical names in the resulting graph (they will be separate but you
have no idea which one is which). This was already an issue
before if you have the same template name in different modules so I
consider this an orthogonal issue.
This fixes the expected failure we already had and I added another
test that checks that colliding template names do at least show up as
separate nodes in the graph. I also manually tested this against
ex-bond-issuance.
Disclaimier: I’m aware that the code is very messy but I tried to
resist the urge to rewrite it completely and only change what was
necessary.
fixes#5776
changelog_begin
- [DAML Compiler] ``damlc visual`` now works properly in projects
consisting of multiple packages.
changelog_end
* Rename templateChoiceId to templateId
changelog_begin
changelog_end
All functions in `DA.Numeric` take the scale of the result as their
first type argument. IMO, this is a nice API since you usually only
want to specify the scale of the result since the scale of the
term arguments is most of the times inferred.
However, the current type signatures in `DA.Numeric` bear quite some
risk of being confusing. For instance, in
```haskell
mul : NumericScale n3 => Numeric n1 -> Numeric n2 -> Numeric n3
```
the naming of the type variables suggests that the order of the
type parameters is `n1 n2 n3` when it actually is `n3 n1 n2`.
I consider the knowledge of implicit `forall`s are filled in quite
expert and hence think we should make the order of these type arguments
explicit.
There is also a related mistake in the docs of `shift`. Running a
scenario confirmed that
```haskell
shift @1 @2 1.0 == 10.0
```
Hence, `shift` has multiplied its argument by `10^(2-1)`, which is
`10^(n1 - n2)`.
CHANGELOG_BEGIN
CHANGELOG_END
This moves the code from the Shake testing module to a separate
module. This is required since the Shake test setup is not really
intended to test multiple packages. I’ve added a new test suite that
makes use of this and has an expected failure for the issue with
multiple packages. The issue is not actually fixed in this PR, I just
wanted to keep the refactoring separate since it is quite large and
noisy.
changelog_begin
changelog_end
