Commit Graph

6545 Commits

Author SHA1 Message Date
Gary Verhaegen
ad94c47605
ci/docker: sync with hub (#9184)
It looks like the last update was not propagated.

CHANGELOG_BEGIN
ChANGELOG_END
2021-03-18 16:01:58 +00:00
Robin Krom
081623b493
fix: bad package id validation (#9183)
Package ids are 64 lower case hex digits.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-18 16:25:57 +01:00
Stefano Baghino
f0c8b1240d
Integrate non-repudiation into HTTP JSON API (#9180)
* Integrate non-repudiation into HTTP JSON API

changelog_begin
changelog_end

Closes #9094

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596400828

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596408663

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596402533

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596624708

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596624174

* Solve compilation error in daml-script

* Address https://github.com/digital-asset/daml/pull/9180#issuecomment-801875959

changelog_begin
- [JSON API] The JAR for the HTTP JSON API is no longer published on
Maven, use the fat JAR as indicated in the documentation.
changelog_end
2021-03-18 15:15:52 +00:00
Andreas Herrmann
dbd8806848
Distinguish all four types of submits (#9179)
changelog_begin
changelog_end

Adds case classes for the four cases
- `submit`
- `submitMulti`
- `submitTree`
- `submitTreeMulti`

Also groups them by single/multi submitter and simple/tree commands
using additional traits.

The logic to classify submits is called earlier to avoid identifying
commands twice, as we did before to correctly identify referenced
contract ids taking `createAndExerciseCmd` into account.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-18 12:54:19 +01:00
Robin Krom
8c64f120da
pkgid data deps (#9153)
* damlc: Allow package IDs in data-dependencies.

This is the next step outlined in issue #8976. If package id's are
present in the `data-dependency` section of the daml.yaml file, we try
to fetch them (and their transitive dependencies) from the default
ledger of the project.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-18 12:17:38 +01:00
Andreas Herrmann
d5ca22815d
Recognize simple createAndExercise commands (#9154)
* Identify simple createAndExercise command

changelog_begin
changelog_end

* Test simple createAndExercise command

* Don't count created and then consumed contracts

* Test nested createAndExercise

* Expand comment about consuming choice

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-18 09:58:34 +00:00
Moritz Kiefer
6c9679f511
Add documentation for the Daml Profiler (#9174)
* Add documentation for the Daml Profiler

changelog_begin

- [Daml Profiler] Daml Connect EE now includes a profiler which can be
used to profile Daml execution time.

changelog_end

* Update docs/source/tools/profiler.rst

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* Make sphinx happy

changelog_begin
changelog_end

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2021-03-18 09:48:10 +01:00
Stephen Compall
a6da995ecb
use representational variance for generated Scala classes (#8879)
* variance calculation without references or cycles

* an idea for handling the recursive reference case elegantly

* extra data for resolving constraints from [recursive] references later

* note on TypeVar case

* trying a different delayed resolution

* quagmire

* the problem: we were trying to solve too early and mix reference levels

* force inner delays to be pulled out; squash the delays iteratively

* make the variance result accessible

* phantoms are covariant

* scalafmt

* simplest GenMap test

* reference tests

* factoring

* don't treat covariant reference arguments as phantom

* making the wrong reference in delays was breaking solve steps

* test cons list self-recursion

* more recursion testing

* -> is better for 2-tuples here

* test cumulative variance tracking

* uninhabited types have variance too

* test mutual recursion with lots of indirect references

* trigger for failing case in incremental resolution

* redesign delayed constraints to handle nesting properly, and be much simpler

* integrate variance calculation with rest of Scala codegen

* handle concrete type and all-phantoms cases; test

* add integration test for variance inference

* add changelog

CHANGELOG_BEGIN
- [Scala Codegen] Type parameters used in GenMap keys are now supported.
  See `issue #8879 <https://github.com/digital-asset/daml/pull/8879>`__.
CHANGELOG_END

* proper doc for covariantVars

- suggested by @cocreature; thanks

* remove useless local var

- suggested by @cocreature; thanks

* add another recursive function to capture more invariants

- suggested by @cocreature; thanks

* absence in what

- suggested by @cocreature; thanks

* use Identifier instead of I

- suggested by @cocreature; thanks

* remove unused type vars; make goSdt cleaner

* also precache variance of choice and key-referenced types

- suggested by @cocreature; thanks
2021-03-17 16:47:18 +00:00
azure-pipelines[bot]
47e0a223e7
rotate release duty after 1.12.0-snapshot.20210316.6523.0.b382fc45 (#9161)
@sofiafaro-da is taking care of 1.12.0-snapshot.20210316.6523.0.b382fc45 (#9160), so they get pushed back to the end of the line.

Please do not merge this before #9160.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-17 14:51:50 +00:00
Gary Verhaegen
1ea00976f0
skip compat tests on release (#9169)
As requested by @coceature.

Note: skipping the ts_lib job is enough to skip all compat tests because
they all depend on it, and in the Azure model if one of your
dependencies was skipped you get skipped too.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-17 13:33:20 +00:00
Sofia Faro
c26c349c8b
Generate exception instances from syntax. (#9140)
* Generate exception instances from syntax.

changelog_begin
changelog_end

* II

* III

* VII

* update ghc patch and add test

* VIII

* IX

* Remove DatatypeContexts

* X

* update stack snapshot

* don't need datatypecontexts warning anymore

* X-2

* XII

* XIII
2021-03-17 12:19:22 +00:00
Moritz Kiefer
9d175ce547
Document profiling internals (#9168)
* Document profiling internals

This is a cleaned up version of my comment on
https://github.com/digital-asset/daml/pull/9151#discussion_r595474708
in the hope that this makes it easier to understand for the next person.

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-03-17 12:18:57 +00:00
Moritz Kiefer
5e97c573e0
Add a test for create and exercise events in the profiler (#9151)
changelog_begin
changelog_end
2021-03-17 12:10:42 +01:00
Moritz Kiefer
75f7688cae
Expose unhandled exception values and types (#9158)
This PR addresses the TODO to expose the actual exception values and
type instead of the message. This allows us to simply speedy a bit by
removing the continuation used for that but more importantly it means
we can now catch and handle the exception in Daml script.

changelog_begin
changelog_end
2021-03-17 12:10:19 +01:00
Gary Verhaegen
e4d5799907
weekly release (#9167)
Retrying #9160.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-17 11:58:14 +01:00
Andreas Herrmann
370f290d3d
Handle createAndExecuteCmd (#9150)
* Identify CreateAndExerciseCommand

changelog_begin
changelog_end

* Add test case for Command.fromTree

* Fix test hierarchy

* SimpleEvent --> SimpleCommand

* encodeSubmitSimpleCommands

* Use encodeCmd in encodeTree

* Use encodeSubmitSimpleCommands for ACS

* Remove unused code

* encodeTree createAndExerciseCmd test case

* treeReferencedCids --> cmdReferencedCids

* Test cmdReferencedCids on createAndExercise

* Handle cid references from createAndExercise correctly

* Rename test suite

* fix indentation

* CreateAndExercise: Only adjacent create & exercise events

Addresses https://github.com/digital-asset/daml/pull/9150#discussion_r595366589

* Add test case for non-adjacent create&exercise

* Scala 2.13

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-17 10:36:53 +00:00
Moritz Kiefer
af0cb30828
Release SDK 1.11.1 (#9162)
changelog_begin
changelog_end
2021-03-17 10:52:56 +01:00
Moritz Kiefer
493e2154fd
Fix flakiness in create-daml-app tests (#9165)
* Fix flakiness in create-daml-app tests

waitForSelector doesn’t do the trick here since one selector is enough
to make this return. We need to wait until we actually have the
expected number. This is what waitForFollowers already did.

changelog_begin
changelog_end

* maybe I should test my code but also I don’t want to

changelog_begin
changelog_end

* fine I tested it

changelog_begin
changelog_end
2021-03-17 09:48:26 +00:00
Gary Verhaegen
17f8bcd37d
fix release job (#9166)
Three issues here:

1. The release job runs on an Azure-hosted agent, so it doesn't have the
   `reset_caches.sh` script (and doesn't need it).
2. The `bash-lib` step should not run if the current job has already
   failed.
3. The `skip-github` jobs should also not run if the job  has failed.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-17 09:46:02 +00:00
Moritz Kiefer
f6effb228a
Add a compile-only damlc target (#9164)
This untangles the dependency structure a bit so that //daml-lf no
longer ends up depending on daml script and sandbox and similar crap
which should improve build times in general.

changelog_begin
changelog_end
2021-03-17 10:31:37 +01:00
Moritz Kiefer
bbea484a11
Alias for conformance tests (#9163)
Not everyone wants to care about LF versions all the time so this PR
adds an alias to the stable version.

changelog_begin
changelog_end
2021-03-17 10:22:27 +01:00
azure-pipelines[bot]
5827ff00c0
release 1.12.0-snapshot.20210316.6523.0.b382fc45 (#9160)
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@sofiafaro-da is in charge of this release.

Commit log:
```
b382fc45ac update areas to alert @S11001001 of changes (#9156)
bbd239b0d3 Unstringify profiler labels (#9147)
3870f84534 Drop profiler from Sandbox CE (#9152)
691edeacf2 ci: fix cache cleanup (#9137)
98c2651998 Support fetching SDK EE tarball in the assistant (#9146)
9b2158508b Add new variant to Value.scala for builtin-exceptions. (#9084)
75cec502ac Add tests for unmangleLenient (#9148)
1df2270cc9 typo (#9145)
6018697fb4 Make async commit configurable and add "LOCAL" async commit option (#9144)
ab90c437a7 update compat versions for 1.11.1-snapshot.20210312.6429.1.7cd6380e (#9135)
498fcc66b4 KV specific LF library (#9100)
c327476da9 Upgrade msys packages (#9139)
089a11443c release job: remove extra bash-lib (#9136)
b0948b417b release 1.11.1-snapshot, take n+1 (#9134)
5dddf0aead Add missing checkout step (#9133)
7669d8c88c daml pkg: split installation of deps and package db inititialization (#9056)
d5e0d6ca00 Release 1.11.1 snapshot (#9130)
7430590c1d Only track cids referenced in root events (#9131)
d4bdb12862 Add pruning stabilization to ledger API changelog (#9126)
ab7e7d9d94 Ignore fetch and lookup-by-key nodes for state update comparison [KVL-854] (#9070)
208f6c1aab Fix javadocs & sources in daml-sdk-head (#9125)
9320a213db [Integration Kit, Ledger API] Remove hint about early access for participant pruning. (#9120)
1c5e64cb82 Fix release notes link in docs (#9122)
ed746976a3 Apply new logo to create-daml-app template (#9105)
7370313bfa Release yet another snapshot (#9118)
```
Changelog:
```

- [Daml Assistant] The assistant now supports an `artifactory-api-key`
  field in `daml-config.yaml`. If you have a license for Daml Connect
  EE you can specify this and the assistant will automatically fetch
  the EE edition which provides additional functionality.

[Integration Kit, Ledger API] Remove hint about early access for participant pruning. It's stable now.
[SDK Assistant] New Daml logo on create-daml-app template
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-03-17 08:38:07 +01:00
Stephen Compall
b382fc45ac
update areas to alert @S11001001 of changes (#9156)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-16 20:25:28 +00:00
Moritz Kiefer
bbd239b0d3
Unstringify profiler labels (#9147)
I don’t entirely buy that we need to use AnyRef at all here given that
the labels are allocated during speedy compilatoin and not runtime but
regardless, we can definitely use proper case classes/objects instead
of arbitrary strings.

changelog_begin
changelog_end
2021-03-16 19:31:11 +01:00
Moritz Kiefer
3870f84534
Drop profiler from Sandbox CE (#9152)
Now that we have an EE edition, we can drop this. Approved by @bame-da

changelog_begin
changelog_end
2021-03-16 18:59:14 +01:00
Gary Verhaegen
691edeacf2
ci: fix cache cleanup (#9137)
This is a continuation of #8595 and #8599. I somehow had missed that
`/etc/fstab` can be used to tell `mount` to let users mount some
filesystems with preset options.

This is using the full history of `mount` hardening so should be safe
enough. The option `user` in `/etc/fstab` automatically disables any kind
of `setuid` feature on the mounted filesystem, which is the main attack
vector I know of.

This works flawlessly on my local VM, so hopefully this time's the
charm. (It also happens to be my third PR specifically targeted on this
issue, so, who knows, it may even work.)

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-16 17:51:38 +01:00
Moritz Kiefer
98c2651998
Support fetching SDK EE tarball in the assistant (#9146)
* Support fetching SDK EE tarball in the assistant

This PR adds an `artifactory-api-key` field to `daml-config.yaml`. If
this is set, we will download the SDK EE tarball if it exists, falling
back to GH releases on 404s (for old releases).

changelog_begin

- [Daml Assistant] The assistant now supports an `artifactory-api-key`
  field in `daml-config.yaml`. If you have a license for Daml Connect
  EE you can specify this and the assistant will automatically fetch
  the EE edition which provides additional functionality.

changelog_end

* Hardcode first EE snapshot

changelog_begin
changelog_end
2021-03-16 14:17:50 +01:00
nickchapman-da
9b2158508b
Add new variant to Value.scala for builtin-exceptions. (#9084)
* Add new variant to Value.scala for builtin-exceptions.

final case class ValueBuiltinException[+Cid](tag: String, value: Value[Cid]) extends Value[Cid]

And push through the code consequences.
Most places fixed up.
A couple more things to do in this PR (marked NICK)
A couple of things which can be left for later (marked 8020)

fix build

fix another scala match

changelog_begin
changelog_end

* fix any match

* add marker of code which needs attending to in the PR

* extend ledger-api value.proto & fix LfEngineToApi

* undo/comment-out the change to value.proto

* add tests in HashSpec for BuiltinException

* code but dont yet enable value-gen for builtin exceptions

* address comments which suggest we crash in various places

* support BuiltinException in scenario_service.proto

* one more TODO 8020 tag
2021-03-16 13:15:33 +00:00
Moritz Kiefer
75cec502ac
Add tests for unmangleLenient (#9148)
A very small first step in testing the profiler.

changelog_begin
changelog_end
2021-03-16 12:38:47 +01:00
Gary Verhaegen
1df2270cc9
typo (#9145)
changelog_begin
changelog_end
2021-03-16 11:30:57 +01:00
Oliver Seeliger
6018697fb4
Make async commit configurable and add "LOCAL" async commit option (#9144)
In replicated Postgres setups, running with SYNCHRONOUS_COMMIT=LOCAL is a
suitable option as it is a trade off to only locally commit synchronously
but not to wait for a network ack.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-16 08:01:01 +01:00
azure-pipelines[bot]
ab90c437a7
update compat versions for 1.11.1-snapshot.20210312.6429.1.7cd6380e (#9135)
update compat versions for 1.11.1-snapshot.20210312.6429.1.7cd6380e

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-03-15 19:24:59 +01:00
Remy
498fcc66b4
KV specific LF library (#9100)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-15 13:33:57 +01:00
Moritz Kiefer
c327476da9
Upgrade msys packages (#9139)
* Upgrade msys2 packages

The old ones pull in dependencies that have disappeared.

changelog_begin
changelog_end

* No special assignment

changelog_begin
changelog_end
2021-03-15 12:08:36 +00:00
Gary Verhaegen
089a11443c
release job: remove extra bash-lib (#9136)
The filesystem is not reset between steps in the same job, so there is
no need to add the `bash-lib` multiple times.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-15 07:53:26 +00:00
Gary Verhaegen
b0948b417b
release 1.11.1-snapshot, take n+1 (#9134)
Bumping the snapshot release number as the fix was on the release
process on main (i.e. not change to the target branch/commit).

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-14 15:14:38 +01:00
Moritz Kiefer
5dddf0aead
Add missing checkout step (#9133)
The revamped release process relies on publish.sh coming from the
release tag. However without the checkout that doesn’t work. This
resulted in us trying to push the EE tarball for a branch that does
not have it yet.

changelog_begin
changelog_end
2021-03-14 13:58:21 +01:00
Robin Krom
7669d8c88c
daml pkg: split installation of deps and package db inititialization (#9056)
* daml pkg: split installation of deps and package db inititialization

This is the next step for the daml package manager program #8976.
This splits the installation of dependencies from the initialization of
the (ghc) package database.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-12 22:55:11 +01:00
Moritz Kiefer
d5e0d6ca00
Release 1.11.1 snapshot (#9130)
* Release 1.11.1 snapshot

See https://github.com/digital-asset/daml/commits/release/1.11.x for
changes. Bugfixes in JSON API, command tracker & docs.

changelog_begin
changelog_end

* bump to include #9132
2021-03-12 20:22:11 +01:00
Andreas Herrmann
7430590c1d
Only track cids referenced in root events (#9131)
* Only cids referenced in root events count

changelog_begin
changelog_end

* Add regression test for inner event cid references

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-03-12 17:55:34 +01:00
Moritz Kiefer
d4bdb12862
Add pruning stabilization to ledger API changelog (#9126)
* Add pruning stabilization to ledger API changelog

changelog_begin
changelog_end

* also can’t type outside of backport

changelog_begin
changelog_end

* I give up

changelog_begin
changelog_end
2021-03-12 16:49:12 +00:00
Miklos
ab7e7d9d94
Ignore fetch and lookup-by-key nodes for state update comparison [KVL-854] (#9070)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-12 15:54:49 +01:00
Moritz Kiefer
208f6c1aab
Fix javadocs & sources in daml-sdk-head (#9125)
The order of the list elements was swapped. I don’t trust myself to
not get this wrong in the future so rather than relying on list order,
I added a proper datatype.

changelog_begin
changelog_end
2021-03-12 15:42:34 +01:00
Gerolf Seitz
9320a213db
[Integration Kit, Ledger API] Remove hint about early access for participant pruning. (#9120)
CHANGELOG_BEGIN
[Integration Kit, Ledger API] Remove hint about early access for participant pruning. It's stable now.
CHANGELOG_END
2021-03-12 13:43:39 +00:00
Moritz Kiefer
1c5e64cb82
Fix release notes link in docs (#9122)
changelog_begin
changelog_end
2021-03-12 13:39:23 +00:00
Stefano Baghino
ed746976a3
Apply new logo to create-daml-app template (#9105)
* Apply new logo to create-daml-app template

changelog_begin
[SDK Assistant] New Daml logo on create-daml-app template
changelog_end

* Fix logo alignment, change capitalization of page title
2021-03-12 13:20:31 +01:00
Moritz Kiefer
7370313bfa
Release yet another snapshot (#9118)
https://www.youtube.com/watch?v=dQw4w9WgXcQ

changelog_begin
changelog_end
2021-03-12 11:58:13 +00:00
Moritz Kiefer
707c86aa93
Fix GCS upload on releases (#9111)
* Fix GCS upload on releases

changelog_begin
changelog_end

* Update azure-pipelines.yml

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-03-12 11:39:56 +01:00
Moritz Kiefer
a3d6c9faf8
Fix artifactory upload path (#9115)
changelog_begin
changelog_end
2021-03-12 11:39:50 +01:00
Gary Verhaegen
c556db48ed
ci/clean-up: remove poweroff (#9108)
It's not working and I can't make it work (see #9096), so I'd rather
just remove it.

CHANGELOG_BEGIN
CHANGELOG_END
2021-03-12 10:48:34 +01:00