Commit Graph

1330 Commits

Author SHA1 Message Date
Moritz Kiefer
81c710f8fc
Add DA.List.BuiltinOrder module (#9226)
This provides variants of `dedup*` and `sort*` which rely on Daml-LF’s
builtin ordering (using Map internally). I don’t have microbenchmarks
but even in macrobenchmarks this is a measurable improvement which
isn’t particularly surprising.

changelog_begin

- [Daml Stdlib] Add `DA.List.Builtinorder` module. This module provides
  variants of `sort*` and `dedup*` which rely on Daml-LF’s builtin
  ordering and are significantly more efficient in some cases.

changelog_end
2021-03-24 16:33:05 +01:00
Moritz Kiefer
4fd1cf7c28
Fix doctests (#9225)
Haven’t tracked down when they broke. They used to work at some point
but atm they are very broken and don’t test anything.

This PR fixes the bash mess to parse the files correctly and fixes two
minor issues that creeped in by virtue of the doctests being broken.

changelog_begin
changelog_end
2021-03-24 12:56:37 +01:00
Moritz Kiefer
90c5ce703a
Optimize mapOptional and add more efficient findOptional (#9214)
* Optimize mapOptional and add more efficient findOptional

Given that we don’t have list fusion catOptionals . map f is clearly
less efficient than this version.

I also added findOptional which can be pretty handy in certain cases.

changelog_begin
changelog_end

* more foldr

changelog_begin
changelog_end

* fix tests

changelog_begin
changelog_end
2021-03-23 17:07:45 +00:00
Remy
1e0c67c306
LF: Add RoundingMode and BigNumeric in Haskell Ast (#9215)
This is part of #8719.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-23 16:50:17 +00:00
Moritz Kiefer
e492603808
More manual list fusion in stdlib (#9217)
Both of these seem sufficiently common that it seems worth optimizing
them.

I realize I’m switching somewhat randomly between foldr and
hand-written recursion. Either should be better than traversing the
list twice but I think it’s a clear sign that we need some benchmarks
to establish the benefits or downsides of one over the other properly.

changelog_begin
changelog_end
2021-03-23 17:08:43 +01:00
Remy
51c5eac66e
LF: Add BigNumeric to archive proto (#9210)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-23 13:58:22 +00:00
nickchapman-da
c1cf2756a8
add rollback node to transactions (#9178)
* New transaction node: NodeRollback. Fixup every match with a crash + TODO.

changelog_begin
changelog_end

* remove 3 methods from NodeRollback which are not needed to fulfill its interface

* add override to remaining 4 methods implemented in body of NodeRollback

* remove unrequired 2nd type parameter (Cid) from NodeRollback

* add missing 8020 marker
2021-03-23 08:49:25 +00:00
Robin Krom
f89aa294e5
damlc pkg: use cache for already present packages (#9193)
* damlc pkg: use cache for already present packages

This only downloads packages from a remote ledger, if the package is not
already present in the dependency db as a normal dependency.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-22 13:57:38 +01:00
Robin Krom
081623b493
fix: bad package id validation (#9183)
Package ids are 64 lower case hex digits.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-18 16:25:57 +01:00
Robin Krom
8c64f120da
pkgid data deps (#9153)
* damlc: Allow package IDs in data-dependencies.

This is the next step outlined in issue #8976. If package id's are
present in the `data-dependency` section of the daml.yaml file, we try
to fetch them (and their transitive dependencies) from the default
ledger of the project.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-18 12:17:38 +01:00
Sofia Faro
c26c349c8b
Generate exception instances from syntax. (#9140)
* Generate exception instances from syntax.

changelog_begin
changelog_end

* II

* III

* VII

* update ghc patch and add test

* VIII

* IX

* Remove DatatypeContexts

* X

* update stack snapshot

* don't need datatypecontexts warning anymore

* X-2

* XII

* XIII
2021-03-17 12:19:22 +00:00
Moritz Kiefer
75f7688cae
Expose unhandled exception values and types (#9158)
This PR addresses the TODO to expose the actual exception values and
type instead of the message. This allows us to simply speedy a bit by
removing the continuation used for that but more importantly it means
we can now catch and handle the exception in Daml script.

changelog_begin
changelog_end
2021-03-17 12:10:19 +01:00
Moritz Kiefer
f6effb228a
Add a compile-only damlc target (#9164)
This untangles the dependency structure a bit so that //daml-lf no
longer ends up depending on daml script and sandbox and similar crap
which should improve build times in general.

changelog_begin
changelog_end
2021-03-17 10:31:37 +01:00
nickchapman-da
9b2158508b
Add new variant to Value.scala for builtin-exceptions. (#9084)
* Add new variant to Value.scala for builtin-exceptions.

final case class ValueBuiltinException[+Cid](tag: String, value: Value[Cid]) extends Value[Cid]

And push through the code consequences.
Most places fixed up.
A couple more things to do in this PR (marked NICK)
A couple of things which can be left for later (marked 8020)

fix build

fix another scala match

changelog_begin
changelog_end

* fix any match

* add marker of code which needs attending to in the PR

* extend ledger-api value.proto & fix LfEngineToApi

* undo/comment-out the change to value.proto

* add tests in HashSpec for BuiltinException

* code but dont yet enable value-gen for builtin exceptions

* address comments which suggest we crash in various places

* support BuiltinException in scenario_service.proto

* one more TODO 8020 tag
2021-03-16 13:15:33 +00:00
Robin Krom
7669d8c88c
daml pkg: split installation of deps and package db inititialization (#9056)
* daml pkg: split installation of deps and package db inititialization

This is the next step for the daml package manager program #8976.
This splits the installation of dependencies from the initialization of
the (ghc) package database.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-12 22:55:11 +01:00
Remy
70379d4e85
LF: Experiment Hook for 1.dev (#9051)
This PR adds a hook in the compiler and the engine to allow easy prototyping of new features.

In particular by using this hook, one can add a new feature in 1.dev with modifying:

    compiler
    type checkers (both Haskell and Scala)
    archive Protobuf
    archive decoder

In addition of development speed, this also adds a bit of confidence, as the peaces enumerated above are not touched when adding the feature: The feature can be added by modifying only 1.dev Daml standard library and Speedy.

aa7991f8 shows a use case of this hook.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-11 21:36:49 +01:00
Sofia Faro
e538f2d676
Add DA.Exception and enable LF primitives. (#9082)
- Add `DA.Exception` module that re-exports `DA.Internal.Exception`
while hiding the `DamlException` class.

- Uncommented out the primitive calls in `DA.Internal.Exception`, so
the standard library is actually calling the underlying LF primitives.

- Expanded the `ExceptionDesugared` test to actually throw and catch
an exception. :-)

changelog_begin
changelog_end
2021-03-11 11:42:16 +00:00
Remy
8a35528514
LF: Make 1.12 default compiler output (#9058)
CHANGELOG_BEGIN
- [Compiler] The compiler produces LF 1.12 by default. LF 1.12 only reduces transaction size.
CHANGELOG_END
2021-03-10 16:44:29 +00:00
Sofia Faro
e6ea8fa9b3
Start adding exception syntax support. (#8960)
* Start adding exception syntax support.

changelog_begin
changelog_end

* .

* ..

* . .

* ....

* .___.

* ..__..

* ._._._.

* update stack snapshot

* recompile ghc-lib

* update stackage

* add CmmParse patch

* incorporate CmmParse.y patch into ghc branch

* Add a first test for exception syntax desugaring.

* Add a test for multiple message declarations

* update ghc patch

* Update duplicate message test with location

* add brackets test

* Add final patch SHA

* update stack-snapshot and unpin for unix

* Update stackage pin on windows
2021-03-10 12:07:53 +00:00
Moritz Kiefer
0281b442b8
-Werror all the things (#9027)
We used to do this for some packages but it broke cpp. I don’t
actually know why it doesn’t do that anymore but I’ll gladly accept
that fact and turn it on everywhere.

changelog_begin
changelog_end
2021-03-04 19:08:59 +01:00
Moritz Kiefer
2c08586d33
Expose Daml stacktraces for Daml Script errors (#9025)
* Expose Daml stacktraces for Daml Script errors

This finally plugs together the pieces from the previous PRs to
provide stacktraces on any ScriptF command (and thereby anything
involving an interaction with the ledger).

fixes #8754

changelog_begin

- [Daml Script] When running Daml Script on the command line you will
  now see a Daml stacktrace on failures to interact with the ledger
  which makes it significantly easier to track down which of the calls
  fails. By default, you will only get the callsite of functions like
  `submit`. To extend the stack trace, add `HasCallStack` constraints
  to functions and those will also be included.

changelog_end

* Fix non-determinism in tests

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-03-04 16:13:50 +01:00
Robin Krom
3758b57a87
pkg deps: Move all dalfs to package dependency directory (#9002)
As a first step for the daml package manager, we move all dalf files
from (data-)dependencies to a new `.daml/dependencies` directory.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-04 11:37:03 +01:00
Moritz Kiefer
e5421288d9
Improve errors on duplicate record field names (#9010)
* Improve errors on duplicate record field names

fixes #8994

changelog_begin
changelog_end

* Apparently I was wrong about names

changelog_begin
changelog_end

* hlint

changelog_begin
changelog_end

* newlines don’t render well in daml build

changelog_begin
changelog_end

* maybe I should test if my code compiles before pushing

changelog_begin
changelog_end
2021-03-04 10:13:24 +01:00
Moritz Kiefer
32dc8b007b
Include stack traces for all ScriptF commands (#8999)
This PR adds stack traces to all ScriptF commands and handles those in
the converter. Those stack traces are not yet used. To ease review,
I’ve left that for a separate PR. The plan is to use this to tackle
https://github.com/digital-asset/daml/issues/8754.

changelog_begin
changelog_end
2021-03-03 15:07:09 +01:00
Robin Krom
7490832966
damlc-test: add 'large' label to BUILD (#8997)
Hopefully this gives the tests more time to complete on the mac nodes.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-03 11:38:57 +00:00
Robin Krom
11ec339f6f
feature: damlc lints all files in project directory (#8979)
* feature: damlc lints all files in project directory

fixes #8887.

This changes the input options of `damlc lint` to take several files. If
no file is given, all `.daml` files contained in the directory are
linted.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-02 15:50:57 +01:00
Moritz Kiefer
ec36066986
Split LedgerInteraction into different files (#8975)
Pure shuffling around there is no logic change in here. I keep getting
lost in the 1.4k LedgerInteraction file so this client splits it up
into a 4 different files to make it a bit easier to navigate.

changelog_begin
changelog_end
2021-03-01 16:27:34 +01:00
Moritz Kiefer
514bf2597e
Include table view for failed scripts (#8967)
fixes #8966

changelog_begin

- [Daml Studio] Failed scripts (and scenarios) now also offer the
  option to view the table view at the state before the failing
  transaction to ease debugging.

changelog_end
2021-03-01 09:58:01 +01:00
Robin Krom
012b04368d
damlc test: feature: add --all flag. (#8919)
* damlc test: feature: add --all flag.

This adds a flag `--all` that will test all present scenarios/scripts in
the code and also report code coverage with respect to all present
templates/choices, whereas without the flag, code coverage is reported
relative to a single package.

CHANGELOG_BEGIN
[damlc test] Added a `--all` flag to test all present scripts/scenarios.
CHANGELOG_END
2021-02-26 18:44:40 +01:00
Moritz Kiefer
275eaedd2a
Fix closure references in Daml Repl (#8950)
* Fix closure references in Daml Repl

Turns out the comment "we probably need to extend this to merge the
modules from each line" was exactly correct: If the result evaluates
to a closure (or a value including a closure), it can reference
definitions from the current module. This happens if the simplifier
lifted something out of the current definition (otherwise we have only
one and it cannot be recursive so no chance of leaking a reference).

This PR fixes this by checking whether the result references the
current module and if it does, we keep it around.

changelog_begin
changelog_end

* Address review comments

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-02-26 13:29:42 +00:00
Gary Verhaegen
1b56b31e80
document potentially-surprising Month <-> Enum (#8923)
Based on a [forum question] by Alxander Bernauer.

[forum question]: https://discuss.daml.com/t/enum-instance-for-month-is-misleading/2098?u=gary_verhaegen

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-25 10:33:52 +01:00
Remy
a9b035e92e
LF: release LF 1.12 (#8856)
CHANGELOG_BEGIN

    - [LF] Release LF 1.12. This version reduce the size of transaction

    - [Compiler]: Change the default LF output from 1.8 to 1.11.

CHANGELOG_END
2021-02-24 21:17:33 +01:00
Gary Verhaegen
f0f2e710c1
document HasField (#8922)
* document HasField

This is stolen directly from @sofiafaro-da's excellent [forum answer].

[forum answer]: https://discuss.daml.com/t/generating-record-update-functions-to-be-used-as-parameters/2091/3?u=gary_verhaegen

CHANGELOG_BEGIN
CHANGELOG_END

* remove currying & add warning to sue dot syntax instead

* add set -euo pipefail to preview script
2021-02-24 16:55:10 +00:00
Moritz Kiefer
8ebc8d3fc8
Sort Daml REPL options to clean up --help (#8936)
Not perfect but probably much cleaner than what we had before. Some
of the orders can definitely be debated and I don’t have strong
feelings beyond the first 5 so if someone feels strongly, I’m open to
changing things.

New output:

Usage: damlc repl [DAR] [-i|--import PACKAGE]
                  [--ledger-host ARG --ledger-port ARG]
                  [--access-token-file TOKEN_PATH] [--application-id ARG]
                  [--tls] [--cacrt ARG] [--pem ARG --crt ARG]
                  [--max-inbound-message-size ARG]
                  [(-w|--wall-clock-time) | (-s|--static-time)]
                  [--project-root ARG] [--project-check]
                  [--include INCLUDE-PATH] [--package-db LOC-OF-PACKAGE-DB]
                  [--shake-profiling PROFILING-REPORT] [--jobs THREADS]
                  [--debug] [--ghc-option OPTION]
  Launch the Daml REPL.

Available options:
  -h,--help                Show this help text
  DAR                      DAR to load in the repl
  -i,--import PACKAGE      Import modules of these packages into the REPL
  --ledger-host ARG        Host of the ledger API
  --ledger-port ARG        Port of the ledger API
  --access-token-file TOKEN_PATH
                           Path to the token-file for ledger authorization
  --application-id ARG     Application ID used for command submissions
  --tls                    Enable TLS for the connection to the ledger. This is
                           implied if --cacrt, --pem or --crt are passed
  --cacrt ARG              The crt file to be used as the trusted root CA.
  --pem ARG                The pem file to be used as the private key in mutual
                           authentication.
  --crt ARG                The crt file to be used as the cert chain in mutual
                           authentication.
  --max-inbound-message-size ARG
                           Optional max inbound message size in bytes.
  -w,--wall-clock-time     Use wall clock time (UTC). (this is the default)
  -s,--static-time         Use static time.
  --project-root ARG       Path to the root of a project containing daml.yaml.
                           You should prefer the DAML_PROJECT environment
                           variable over this option.See
                           https://docs.daml.com/tools/assistant.html#running-commands-outside-of-the-project-directory
                           for more details.
  --project-check          Check if running in Daml project.
  --include INCLUDE-PATH   Path to an additional source directory to be included
  --package-db LOC-OF-PACKAGE-DB
                           use package database in the given location
  --shake-profiling PROFILING-REPORT
                           Directory for Shake profiling reports
  --jobs THREADS           The number of threads to run in parallel. When -j is
                           not passed, 1 thread is used. If -j is passed, the
                           number of threads defaults to the number of
                           processors. Use --jobs=N to explicitely set the
                           number of threads to N. Note that the output is not
                           deterministic for > 1 job.
  --debug                  Enable debug output
  --ghc-option OPTION      Options to pass to the underlying GHC

changelog_begin
changelog_end
2021-02-24 14:31:55 +00:00
Remy
d17dd7e5f1
Ledger API test tool: test against legacy and preview version. (#8913)
* Ledger API test tool: test against legacy and preview version.

+ use dictionary instead of alias to map version keywords to LF version 

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-23 09:19:17 +01:00
Remy
d92f2c7003
Ledger: refactor bazel packaging of tedger test tool (#8894)
improve previous generalization from #8695

- use lf version instead keyword (like 'stable', 'latest', 'dev') to
  tag actual target.  This will allow two keywords to map to the same
  versions without doing twice the compilation/test work.

- use alias to map keywords tag target to versioned tag target.

- move package manage dar to test_commong.

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-22 11:41:19 +01:00
Sofia Faro
ff7f54b896
Rename tryCatchRaw to _tryCatch. (#8897)
This matches the name of other internal functions more closely, like
_toAnyTemplate and _fromAnyChoice.

changelog_begin
changelog_end
2021-02-18 18:26:08 +00:00
Jost Berthold
5d44ff9085
Upgrade the LF library build setup (resolver, dependencies, version) (#8865)
Upgrade Daml LF library extraction to latest versions and newer GHC

changelog_begin
changelog_end
2021-02-17 07:07:17 +01:00
Moritz Kiefer
777749539c
Draw the rest of the Scala 2.13 owl (#8852)
* Draw the rest of the Scala 2.13 owl

Not quite but pretty close and this switches us over from inclusions
to exclusions which makes it much easier to track.

Ledger API test tool should be fixed by #8821. Non-repudiation needs a
tiny bit of work since unwrapArray doesn’t work the same on 2.13 but
shouldn’t be hard to fix.

changelog_begin
changelog_end

* Fix ScriptService tests

Those tests were all dumb. They asserted on a fixed order while the
function to sort the things was broken so we ended up with the random
Map order which is unsurprisingly not the same.

This is easily fixed by fixing the sort function.

There is also a second issue with query not sorting.

changelog_begin
changelog_end

* Turns out if you fix one test the next one breaks

And clearly nobody ever tested this or give this a second thought.

changelog_begin
changelog_end
2021-02-15 19:20:24 +00:00
Robin Krom
0c72d07f27
damlc: fix: catch UserError instead IOException (#8850)
* damlc: fix: catch UserError instead  IOException

When the package-db metadata can not be parsed, an exception is thrown
and needs to be catched.  Previously we were catching IOException, but
in this case it's actually a UserError that is thrown.

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-15 19:16:37 +01:00
Robin Krom
5ec36e4ff9
damlc test: show full coverage report (#8834)
* damlc test: show full coverage report

We add a flag `show-coverage` to `damlc-test` to show templates that
are never created and choices that are never executed during the tests.

CHANGELOG_BEGIN
[damlc test] A new flag `show-coverage` shows full test coverage
reports.
CHANGELOG_END
2021-02-15 16:40:13 +01:00
nickchapman-da
dbd017ee49
Support exceptions in speedy. (#8612) 2021-02-15 11:07:12 +00:00
Sofia Faro
34fe42237d
data-deps: Dont let defaults leak constraints. (#8833)
* data-deps: Dont let defaults leak constraints.

Fixes #8802

changelog_begin
changelog_end

* use expandSynApp instead of expandTypeSynonyms

* revert isConstraint change

* fix indentation
2021-02-12 15:44:29 +00:00
Remy
4d38719cb2
LF: first draft of the exception internal API for PartialTransaction (#8808)
This is part of #8020

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-12 14:06:34 +01:00
Moritz Kiefer
68bd6f065c
Fix warnings when compiling daml-stdlib (#8826)
No point in outputting a bunch of warnings about deprecations that are
expected and cannot be fixed.

changelog_begin
changelog_end
2021-02-12 13:10:39 +01:00
Moritz Kiefer
f15a2a36b0
Bump DLint to include fix for disable comments (#8820)
changelog_begin

- [DLint] You can now disable dlint hints for a specific functions in
  the form `{- DLINT ignore functionName "hintName" -}`, e.g.,

```
{- DLINT ignore noHint "Use concatMap" -}
noHint f xs = concat (map f xs)
```

Note: The undocumented {- HLINT -} pragmas are no longer supported.

changelog_end
2021-02-12 11:13:44 +01:00
Robin Krom
49d957bec1
damlc: show test coverage (#8810)
* damlc: show test coverage

We add output to show percentage of created templates and executed
choices for `damlc test`.

This fixes #6370.

CHANGELOG_BEGIN
[damlc] Feature: Test coverage is reported for `damlc test`.
CHANGELOG_END

* added a test

* Update compiler/damlc/lib/DA/Cli/Damlc/Test.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* suggestions

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-02-11 17:08:41 +01:00
Remy
c5f0b3636c
LF: Refactor PartialTransaction Context (#8804)
This prepares the introduction of rollback node.

This is part of #8020.

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-10 14:12:57 +01:00
Moritz Kiefer
e196a7cc73
Clarify documentation around --project-root (#8790)
fixes #8783

changelog_begin
changelog_end
2021-02-10 12:03:59 +01:00
Sofia Faro
2943c4089c
Pick up user-defined exceptions in LFConversion. (#8777)
This uses a "DamlException" tag to annotate which types should
be exported as exceptions. This tag will be added during the
desugaring of exceptions.

The code that scrapes for the HasMessage instance is based on the
corresponding template instance scraping code, but simplified since
we only have to pick up one instance. If we decide to pack more
information into the DefException structure in the future, we can
easily extend this.

This PR also adds a small test to make sure that a user-defined
exception is exported as such.

changelog_begin
changelog_end
2021-02-09 12:27:09 +00:00