Commit Graph

6642 Commits

Author SHA1 Message Date
Moritz Kiefer
0f1d2a9c47
Upgrade akka to 2.6.13 (#9026)
Seems to have a few useful bugfixes
https://akka.io/blog/news/2021/02/23/akka-2.6.13-released. Unclear if
we’re affected by those bugs but I’d rather not worry about whether we are.

changelog_begin
changelog_end
2021-03-05 10:31:37 +01:00
Moritz Kiefer
b1a59d0c70
Release SDK 1.11.0 RC 2 (#9031)
Only change is d3d5042ac0

changelog_begin
changelog_end
2021-03-04 20:07:41 +00:00
Moritz Kiefer
41e049339b
Fix --extract option in ledger api test tool (#9029)
This broke in #8894

changelog_begin
changelog_end

windows is great

changelog_begin
changelog_end
2021-03-04 18:32:18 +00:00
Moritz Kiefer
0281b442b8
-Werror all the things (#9027)
We used to do this for some packages but it broke cpp. I don’t
actually know why it doesn’t do that anymore but I’ll gladly accept
that fact and turn it on everywhere.

changelog_begin
changelog_end
2021-03-04 19:08:59 +01:00
Moritz Kiefer
2c08586d33
Expose Daml stacktraces for Daml Script errors (#9025)
* Expose Daml stacktraces for Daml Script errors

This finally plugs together the pieces from the previous PRs to
provide stacktraces on any ScriptF command (and thereby anything
involving an interaction with the ledger).

fixes #8754

changelog_begin

- [Daml Script] When running Daml Script on the command line you will
  now see a Daml stacktrace on failures to interact with the ledger
  which makes it significantly easier to track down which of the calls
  fails. By default, you will only get the callsite of functions like
  `submit`. To extend the stack trace, add `HasCallStack` constraints
  to functions and those will also be included.

changelog_end

* Fix non-determinism in tests

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-03-04 16:13:50 +01:00
Andreas Herrmann
c7ee410fa3
Use DA.Map in triggers if available (#9023)
* Use DA.Map in triggers if available

Replaces DA.Next.Map by DA.Map in the trigger library if the DAML-LF
version supports it, i.e. above 1.11.
Selects on `DAML_GENMAP` and `DAML_GENERIC_COMPARISON` using `CPP`.

changelog_begin
- [Triggers] The trigger library now uses `DA.Map` instead of the
  deprecated `DA.Next.Map` if the targeted Daml-LF version supports it.
  This is a breaking change: Code that interfaced with the triggers
  library using `DA.Next.Map`, e.g. with
  `Daml.Trigger.getCommandsInFlight` or `Daml.Trigger.Assert.testRule`,
  will need to be changed to use `DA.Map` instead.
changelog_end

* Deduplicate GMap imports in triggers

* DA.Next.Map --> DA.Map in carbon-upgrade-trigger

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-04 14:15:01 +01:00
Stefano Baghino
d347934db3
Make non-repudiation PostgreSQL back-end certificate adding idempotent (#9024)
Closes #9021

changelog_begin
changelog_end
2021-03-04 13:26:39 +01:00
Stefano Baghino
eaf7fb6f63
Add non-repudiation API (#8982)
* Add non-repudiation API

The Web API for the non-repudiation proxy allows to retrieve the signed
payload for a given command, add a new certificate or query for an
existing one.

Closes #8658

changelog_begin
changelog_end

* Fix Scala 2.13 compilation error

* Test error conditions

* Add tests for errors and logging

* Fix Scala 2.13 compilation error

* Address https://github.com/digital-asset/daml/pull/8982#discussion_r585641768

* Address https://github.com/digital-asset/daml/pull/8982#discussion_r585674885

* Address https://github.com/digital-asset/daml/pull/8982#discussion_r585668733
2021-03-04 11:02:25 +00:00
Stefano Baghino
5a3c95251c
Document latency of Windows ad-hoc machine software installation (#9022)
changelog_begin
changelog_end
2021-03-04 10:41:21 +00:00
Robin Krom
3758b57a87
pkg deps: Move all dalfs to package dependency directory (#9002)
As a first step for the daml package manager, we move all dalf files
from (data-)dependencies to a new `.daml/dependencies` directory.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-04 11:37:03 +01:00
azure-pipelines[bot]
f70d3391ef
update compat versions for 1.10.2 (#9016)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-04 11:36:11 +01:00
Moritz Kiefer
e5421288d9
Improve errors on duplicate record field names (#9010)
* Improve errors on duplicate record field names

fixes #8994

changelog_begin
changelog_end

* Apparently I was wrong about names

changelog_begin
changelog_end

* hlint

changelog_begin
changelog_end

* newlines don’t render well in daml build

changelog_begin
changelog_end

* maybe I should test if my code compiles before pushing

changelog_begin
changelog_end
2021-03-04 10:13:24 +01:00
Moritz Kiefer
65fbcfe8e9
Move execution of commands out of Runner (#9009)
This PR extracts the implementation of the individual commands out of
Runner and into the commands. Turns out, object oriented programming
is good actually.

This is another prerequisite for including stack traces. With this PR
landed, it should be quite easy to adapt Runner to rethrow failed
futures including the daml stacktrace.

changelog_begin
changelog_end
2021-03-04 10:07:36 +01:00
Gary Verhaegen
2688ad6f0d
autorelease: improved PR message (#9008)
Thanks to @stefanobaghino-da for the suggestion.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-03 21:16:23 +00:00
Moritz Kiefer
185e91646d
Release 1.10.2 (#9006)
This includes
80d080ef9f (and
only that compared to 1.10.1.

changelog_begin
changelog_end
2021-03-03 19:31:53 +01:00
mziolekda
4ed3f1a6de
Cut a snapshot release for the 1.8.0 special patch (#8998)
* cut a snapshot release for the 1.8.0 special patch

CHANGELOG_BEGIN
CHANGELOG_END

* update for #9000

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-03-03 18:44:59 +01:00
azure-pipelines[bot]
ba1456fd5a
rotate release duty after 1.11.0-snapshot.20210302.6414.0.72870630 (#8992)
@stefanobaghino-da is taking care of 1.11.0-snapshot.20210302.6414.0.72870630 (#8991), so they get pushed back to the end of the line.

Please do not merge this before #8991.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-03 17:52:24 +01:00
Gary Verhaegen
259bf682c4
compat: fix killing stale sandbox(es) (#9003)
We've seen this happen once; I don't think it's worth investigating.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-03 15:14:05 +00:00
Moritz Kiefer
32dc8b007b
Include stack traces for all ScriptF commands (#8999)
This PR adds stack traces to all ScriptF commands and handles those in
the converter. Those stack traces are not yet used. To ease review,
I’ve left that for a separate PR. The plan is to use this to tackle
https://github.com/digital-asset/daml/issues/8754.

changelog_begin
changelog_end
2021-03-03 15:07:09 +01:00
azure-pipelines[bot]
1525957c7a
release 1.11.0-snapshot.20210302.6414.0.72870630 (#8991)
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.

@stefanobaghino-da is in charge of this release.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-03 14:29:41 +01:00
Andreas Herrmann
926949e503
Use single party submit/submitTree where appropriate (#8995)
* Move [] into submit

* Use submit instead of submitMulti for single party

changelog_begin
changelog_end

* Test multi-party submission

* Factor out construction of submit command

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-03 14:16:48 +01:00
Stefano Baghino
145ddaa86d
Point out where the documentation for release.sh is (#8996)
changelog_begin
changelog_end
2021-03-03 11:55:30 +00:00
Robin Krom
7490832966
damlc-test: add 'large' label to BUILD (#8997)
Hopefully this gives the tests more time to complete on the mac nodes.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-03 11:38:57 +00:00
Kamil Bożek
32d4bf92ec
Do not require a JWT token for Health and Reflection services [DPP-277] (#8969)
* Do not require a JWT token for Health and Reflection services

CHANGELOG_BEGIN
- A JWT token is no longer required to call methods of Health and Reflection services
CHANGELOG_END

* Let service's authorizer decide about rejections

* Updated authorization test

* Added integration test for unsecured authorisation test for the Health service

* Added integration test for unsecured authorisation test for the Server Reflection service

* Updated Claims doc comments

* Minor change

* Reduced code duplication with SecuredServiceCallAuthTests and UnsecuredServiceCallAuthTests

* Added copyrights

* Move response status handling logic to Authorizer
2021-03-03 11:05:35 +00:00
Moritz Kiefer
a40b4a978e
Factor out "parsing" of ScriptF constructors (#8984)
I was trying to add stacktraces and this giant pattern match got
really annoying so this PR tries to improve the situation somewhat by
factoring out the parsing into a separate module. There is more stuff
we can do here but this is at least a start.

changelog_begin
changelog_end
2021-03-03 11:05:51 +01:00
Andreas Herrmann
f4eeed396a
Update rules_haskell (#8987)
changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-03 10:40:26 +01:00
Gerolf Seitz
fc743374f6
[participant] Proper async boundaries between pipeline stages (#8993)
mapAsync doesn't introduce async boundaries, meaning there's not much of
a pipelining going on.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-03 09:29:51 +00:00
Moritz Kiefer
4001f5c9bd
Conduitify download in release file (#8988)
* Conduitify download in release file

Apologies, my ocd kicked in when I saw this in another PR.

changelog_begin
changelog_end

* fixup deps

changelog_begin
changelog_end

* i should stop working

changelog_begin
changelog_end
2021-03-03 08:49:41 +01:00
Moritz Kiefer
7287063072
Fix protobuf structure for empty import prefix (#8986)
052f69cde9
broke the structure of our protobufs by changing the import prefix
from "." to an empty string which ends up flattening the whole
file. This PR changes the default for tars to match the old
behavior. We can’t just change the default at the function level since
the prefix is used outside of pkg_tar and in those places an empty
string is required :(

changelog_begin
changelog_end
2021-03-02 17:26:34 +01:00
Robin Krom
11ec339f6f
feature: damlc lints all files in project directory (#8979)
* feature: damlc lints all files in project directory

fixes #8887.

This changes the input options of `damlc lint` to take several files. If
no file is given, all `.daml` files contained in the directory are
linted.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-02 15:50:57 +01:00
Gary Verhaegen
9da40ea426
ci/cron: fix retry policy (#8985)
I think the retry is clobbering the files. Here is my theory:
- The HTTP request is lazy, i.e. it starts producing a byte stream
  before it has finished downloading.
- The connection somehow crashes in the middle of that lazy handling,
  possibly because the Haskell code blocks for too long on something
  else and GCP thus closes the connection. (If this is true, making sure
  we download the entire thing before we start writing may make the
  download more reliable.) This explains why we get a "resource vanished"
  and not a plain 404 to start with.
- The retry policy doesn't know anything about HTTP requests; it just
  sees an IO action throwing an exception and restarts the whole thing.
- Because the IO action opens the file in Append mode, we thus end up
  with a file that is too big and has its "starting bytes" multiple
  times. That obviously fails to sign-check.

If this is what happens then the retry does not help at all, which does
seem to be what we've been observing (though I haven't tracked the exact
error rate too closely). The fix would likely be as simple as changing
`IO.AppendMode to IO.WriteMode (which truncates, per [documentation]).

[documentation]: https://hackage.haskell.org/package/base-4.14.1.0/docs/System-IO.html

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-02 13:51:00 +00:00
Andreas Herrmann
65101e37d0
Batch creation of ACS (#8978)
* Implement batching of ACS submits

* Configure ACS batch size

changelog_begin
changelog_end

* Add tests for ACS batching

* Avoid empty lines in dump

* Stream --> LazyList

* use grouped

* Bump default ACS batch size to 10

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-02 14:00:03 +01:00
Remy
fea1ab0c5e
KV: fix transaction deserialization (#8958)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-02 11:43:55 +01:00
Stefano Baghino
0b3758eda0
Move non-repudiation in-memory backend to testing module (#8981)
This will allow to share it with the non-repudiation API tests.

changelog_begin
changelog_end
2021-03-01 18:52:51 +01:00
Moritz Kiefer
707e508f62
Bump JdbcLedgerDao resource acquisition timeout (#8980)
I’ve seen this timeout several times on CI. It seems somewhat expected
that migrations get slower as we add more, so I don’t think this is
necessarily worth investigating but if someone wants to do so, be my
guest. I don’t know enough about those tests to do so myself.

changelog_begin
changelog_end
2021-03-01 15:51:19 +00:00
Moritz Kiefer
ec36066986
Split LedgerInteraction into different files (#8975)
Pure shuffling around there is no logic change in here. I keep getting
lost in the 1.4k LedgerInteraction file so this client splits it up
into a 4 different files to make it a bit easier to navigate.

changelog_begin
changelog_end
2021-03-01 16:27:34 +01:00
Gerolf Seitz
141b34b15d
Let migrations not explode memory (#8883)
* Avoid a match error when upgrading LF values

* Always set the fetch-size for queries in migrations

Not doing so causes the JDBC driver to try to load all rows into memory.
This will result in an out of memory error on any reasonably sized ledger.

* Fix compile errors

CHANGELOG_BEGIN
[Integration Kit, Daml on SQL] Fix potential out of memory issue when running
migrations on an existing index DB with many events/contracts/transactions.
CHANGELOG_END
2021-03-01 14:36:08 +01:00
Moritz Kiefer
576d0cb758
Bump Ledger API version for LF 1.12 (#8972)
changelog_begin
changelog_end
2021-03-01 12:42:33 +00:00
Moritz Kiefer
71116b053c
Add a Ledger API changelog (#8974)
* Add a Ledger API changelog

changelog_begin
changelog_end

* Update docs/source/support/compatibility.rst

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2021-03-01 12:42:27 +00:00
azure-pipelines[bot]
1da3e64f0c
update NOTICES file (#8968)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-01 10:49:13 +01:00
Moritz Kiefer
514bf2597e
Include table view for failed scripts (#8967)
fixes #8966

changelog_begin

- [Daml Studio] Failed scripts (and scenarios) now also offer the
  option to view the table view at the state before the failing
  transaction to ease debugging.

changelog_end
2021-03-01 09:58:01 +01:00
Moritz Kiefer
3f4cebd808
Fix daml-sdk-head Maven install (#8962)
* Fix daml-sdk-head Maven install

There were two issues:

1. The generated pom.xml used to install everything in one step
referenced the source files in the bazel directory rather than the
released artifacts. This luckily worked so far but falls apart now
since the 2.13 build overwrites the 2.12 build. Same issues that also
blew up the actual release on Thursday.

2. Two artifacts are published as shaded jars. We treated those like
Scala artifacts while they should be treated more like fat
jars. They’re not quite like those since the file paths are different
so I gave them a new release type.

changelog_begin
changelog_end

* shut up hlint

changelog_begin
changelog_end

* Fix broken comment for jarjar

changelog_begin
changelog_end
2021-02-26 21:20:04 +01:00
Robin Krom
012b04368d
damlc test: feature: add --all flag. (#8919)
* damlc test: feature: add --all flag.

This adds a flag `--all` that will test all present scenarios/scripts in
the code and also report code coverage with respect to all present
templates/choices, whereas without the flag, code coverage is reported
relative to a single package.

CHANGELOG_BEGIN
[damlc test] Added a `--all` flag to test all present scripts/scenarios.
CHANGELOG_END
2021-02-26 18:44:40 +01:00
Oliver Seeliger
e5d12e2875
Upgrade canton to 0.21.0 which supports lf-version 1.11 (#8963)
Updated crypto config and removing two sets of tests that had been
disabled against canton 0.19.0

CHANGELOG_BEGIN
CHANGELOG_END
2021-02-26 16:52:03 +00:00
Andreas Herrmann
ddf8e13705
Helpers to create test ACS and TransactionTree (#8961)
* Helpers to create test ACS and TransactionTree

changelog_begin
changelog_end

* use mutable count

c62fff7921 (r583743837)

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-02-26 16:41:36 +00:00
Moritz Kiefer
275eaedd2a
Fix closure references in Daml Repl (#8950)
* Fix closure references in Daml Repl

Turns out the comment "we probably need to extend this to merge the
modules from each line" was exactly correct: If the result evaluates
to a closure (or a value including a closure), it can reference
definitions from the current module. This happens if the simplifier
lifted something out of the current definition (otherwise we have only
one and it cannot be recursive so no chance of leaking a reference).

This PR fixes this by checking whether the result references the
current module and if it does, we keep it around.

changelog_begin
changelog_end

* Address review comments

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-02-26 13:29:42 +00:00
Stefano Baghino
929e0c12d4
Temporarily cut compatibility tests with incompatible versions of Daml-LF (#8959)
* Temporarily cut compatibility tests with incompatible versions of Daml-LF

This adds more tests to the exclusions added as part of #8957

All SDK/platform compatibility tests are not filtered to use compatible
Daml-LF versions, not just those involving the Ledger API Test Tool.

changelog_begin
changelog_end

* Fix Bazel code
2021-02-26 13:47:59 +01:00
Andreas Herrmann
4fcd2b833f
Avoid submitTree for a sequence of creates (#8952)
* use scalaz.std.iterable

* Use submitMulti for sequence of creates

Instead of submitMultiTree, which is not necessary in such cases.
Avoids redundant binds, i.e. when we only bind one result and it is from
the last action.

changelog_begin
changelog_end

* Extend EncodeTreeSpec

* Factor out encoding created events

* Factor out encodeSubmitCreatedEvents

* Remove redundant encodeSubmitCreatedEvent

* Test encoding multiple creates

* Drop redundant case

* Fix duplicate created event

* Avoid .last

* Use traverse

* Avoid silently discarded return values

* mkCreate helper in EncodeTreeSpec

* mkExercised helper in EncodeTreeSpec

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-02-26 11:29:45 +00:00
Stefano Baghino
72951b8416
Temorarily cut compatibility tests with incompatible versions of Daml-LF (#8957)
These cause spurious failures on CI.

As described in the code, the proper fix is to use the correct Daml-LF version for every pair.

changelog_begin
changelog_end
2021-02-26 09:55:43 +00:00
Moritz Kiefer
623f4e27b7
Commit compatibility/.bazelrc (#8955)
We initially did not commit this since keeping it in sync is
annoying. We cannot use a symlink because Windows. However, it changes
sufficiently rarely that checking it in + a check that the two are in
sync seems less fragile than the current state.

changelog_begin
changelog_end
2021-02-25 19:53:30 +01:00