Commit Graph

5923 Commits

Author SHA1 Message Date
Gary Verhaegen
604787eb07
fix Slack notifications (#8327)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-17 14:32:49 +01:00
Robert Autenrieth
f3d8f05070
Enable multi-party submissions [KVL-708] (#8266)
* Remove single-party check

CHANGELOG_BEGIN
- [Ledger API] The ledger API now supports multi-party submissions.
  In order to use multi-party submissions, use the new act_as and
  read_as fields in submission requests.
CHANGELOG_END

* Remove usage of temporary SubmitterInfo methods

* Fix validator tests

* Fix auth test

* Add multi-party tests to the ledger API

* Fix compile errors

* Improve tests

* Remove temporary single-party method from SubmitterInfo

* Remove temporary single-party method from SubmitterInfo companion object

* Remove temporary single-party method from TxEntry

* Run multi-party submission ITs for ledger-on-memory and ledger-on-sql

* Minor improvement

Co-authored-by: Kamil Bozek <kamil.bozek@digitalasset.com>
2020-12-17 13:42:39 +01:00
Moritz Kiefer
b32789025e
Reset Windows cache (#8326)
changelog_begin
changelog_end
2020-12-17 12:24:07 +00:00
Moritz Kiefer
a8776b0ba1
Fix documentation for InclusiveFilters (#8322)
As discussed in
https://discuss.daml.com/t/registeredtemplates-behaves-as-allindar/1789/5?u=cocreature
the current behavior doesn’t match the docs. There doesn’t seem a good
usecase to change the existing behavior so adapting the docs seems
more sensible.

changelog_begin
changelog_end
2020-12-17 08:33:35 +01:00
Jost Berthold
04a6d4c86c
HOTFIX bump versions for Haskell LF libraries (#8287)
* HOTFIX update LF lib extraction to 0.1.8.0 + add new things

* HOTFIX add a dummy project file for testing

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-17 17:41:09 +11:00
Robin Krom
fa9b470249
install script: check available disk space (#8317)
* install script: check available disk space

This fixes #6993. We check that /tmp offers enough space to extract the
sdk tarball and fail otherwise. An alternative extraction directory can
be set by setting the TEMPDIR environment variable.

CHANGELOG_BEGIN
CHANGELOG_END

* don't remove user specified directory

CHANGELOG_BEGIN
CHANGELOG_END

* Update daml-assistant/get-daml.sh

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2020-12-16 18:15:01 +01:00
Samir Talwar
021697fff7
kvutils: Remove fingerprints from the pre-executing validator. [KVL-747] (#8296)
* kvutils: Remove fingerprints from the pre-executing validator.

The submission validator doesn't care about fingerprints, it only needs
to know about them in order to discard them from the state value. This
introduces a new typeclass, `HasDamlStateValue`, which can be customized
for various state value types to make submission validation generic.

A little more code for now, but we're setting things up so it can be
deleted later.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Add more comments for `PreExecutingSubmissionValidator`.
2020-12-16 15:15:08 +00:00
nickchapman-da
2d511b449d
Make choice observers mandatory when available. (#8316)
* Make choice observers mandatory when available.

This is an invariant of the DAML-LF proto.

The haskell AST allows missing choice observers for any version, because the AST represents what was written in the syntax. However, if we are targeting a new DAML-LF version (>= `featureChoiceObservers`) then `Nothing` is converted to `Just (ENil TParty)` by the haskell encoder (`EncodeV1.hs`) to satisfy the proto invariant.

On the scala side, the decoder (`DecodeV1.scala`) now insists that choice observers are present when_ the version is new. (>= `LV.Features.choiceObservers`).

changelog_begin
changelog_end

* LF: LF encoder generate always choice observer if lf >= 1.dev (#8318)

This is a follow up of #8316.
This advances the state of #7709

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-12-16 14:38:25 +00:00
Gary Verhaegen
7a5940d076
ts: generated code includes Template.QueryResult (#8315)
See [discourse].

[discourse]: https://discuss.daml.com/t/best-practice-for-getting-ledger-state-in-multiple-nested-react-components/1814/3

CHANGELOG_BEGIN

* [JavaScript Client Libraries] When using our React wrapper from
  TypeScript, users could end up having to manually reconstruct a
  `QueryResult` type for a specific template, leading to code looking
  like (assuming a `User` template):
  ```
  const allUser: QueryResult<User.User, User.User.Key, typeof User.User.templateId> = useStreamQueries(User.User);
  ```
  Our codegen will now generate an additional type definition per
  template such that the above can be rewritten as
  ```
  const allUser: User.User.QueryResult = useStreamQueries(User.User);
  ```
  Developers using our JavaScript bindings directly (i.e. without taking
  advantage of the TS type definitions) are not affected. Existing
  (long-form) type declaractions will also keep working.

  Note: In order to make this change, we had to move the definition of
  the `QueryResult` type from the `@daml/react` package to the
  `@daml/ledger` package. However, the `@daml/react` package is still
  exporting it as before, so this is a backwards-compatible change.

CHANGELOG_END
2020-12-16 14:31:36 +01:00
azure-pipelines[bot]
5073f56595
rotate release duty after 1.8.0-snapshot.20201215.5907.0.a6ed34c5 (#8307)
@sofiafaro-da is taking care of 1.8.0-snapshot.20201215.5907.0.a6ed34c5 (#8306), so they get pushed back to the end of the line.

Please do not merge this before #8306.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-16 13:12:06 +00:00
curiousleo-da
edae9ddb33
Add spans for index DB reads [LPF-270] (#8239)
CHANGELOG_BEGIN
- [Integration Kit] Emit OpenTelemetry spans and events for index DB
  reads
CHANGELOG_END
2020-12-16 12:39:24 +01:00
Moritz Kiefer
fbf1820f94
Fix slack escaping (#8311)
The previous escaping syntax doesn’t seem to work (after the first
layer you no longer have a variable) and has broken our build
reporting completely.

changelog_begin
changelog_end
2020-12-16 11:18:41 +00:00
Moritz Kiefer
1f048af9d8
Release 1.9 snapshot (#8308)
changelog_begin
changelog_end
2020-12-16 10:41:26 +00:00
Remy
fe44764e61
LF: Refactor TransactionVersion (#8299)
Simplify transaction version. In particular we drop the dependency to LfVersions abstract class.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-16 11:27:54 +01:00
Moritz Kiefer
dd12f03147
Release SDK 1.8 (#8309)
changelog_begin
changelog_end
2020-12-16 09:46:55 +01:00
Andreas Herrmann
652c9428ed
Use execution context, actor system, and materialzer from request context in route (#8300)
* Extract actor system and execution context from request context

* Extract actor system and execution context from request context

* Extract actor system, materializer, and execution context from request context

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-16 09:41:42 +01:00
Miklos
a6ed34c5d3
Release 1.6.1-snapshot.20201215.5318.0.547abc97. (#8304)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-15 21:32:22 +01:00
Gary Verhaegen
e28699dbb0
fix Slack messages (#8292)
When including commit titles in Slack messages, care has to be taken to
escape what Slack considers [control characters], namely `&`, `>` & `<`.

[control characters]: https://api.slack.com/reference/surfaces/formatting#escaping

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-15 18:11:11 +01:00
Gary Verhaegen
072c985328
accept new nix key (#8298)
We're rotating the nix key used by CI to sign our artifacts. This is step
1: add the new public key.

Step 2 will be to update the CI configuration to use the new key, and
step 3 will be the removal of the old key from this conf file.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-15 16:33:47 +01:00
Brian Healey
e29a09dc89
CODEOWNERS for NOTICES file change (#8297)
CHANGELOG_BEGIN
CHANGELOG_ENDs

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
2020-12-15 14:47:39 +00:00
azure-pipelines[bot]
bac8cd04ff
update NOTICES file (#8288)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-15 14:41:12 +00:00
Samir Talwar
54687a579c
kvutils: Reduce the surface area of fingerprints in pre-execution. [KVL-747] (#8289)
* kvutils: Move read set generation to the commit strategy.

* kvutils: Make the submission validator abstract over the read set.

* kvutils: Split post-execution conflict detection from finalization.

* kvutils: Ignore the read set in the post-execution finalizer.

* kvutils: Extract a trait for conflict detection.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Extract out a trait for post-execution finalization.

* kvutils: Generalize the write set in PreExecutingValidatingCommitter.

* kvutils: In submission validation, create the input state once.

* kvutils: Generalize PreExecutingCommitStrategy over keys and values.

* kvutils: Extract KeyNotPresentInInputException to the top level.

* kvutils: Inline type aliases where they're only used once or twice.

* kvutils: Improve naming and commentary around post-execution.
2020-12-15 12:52:08 +00:00
Martin Huschenbett
3c5191acb8
damlc: Don't warn on ExplicitNamespaces extension (#8285)
* damlc: Don't warn on ExplicitNamespaces extension

Since we allow the `TypeOperators` extension, which implies
`ExplicitNamespaces`, there is no point in warning in warning about
the latter.

There's not much to test for this wrt to `data-dependencies` since we
don't reconstruct export lists.

CHANGELOG_BEGIN
damlc: Don't warn on ExplicitNamespaces extensions anymore.
CHANGELOG_END

* Fix a test expecting a warning about ExplicitNamespaces

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-15 12:10:38 +00:00
Stefano Baghino
6580dbd15c
Benchmarks for Speedy/DAML-LF/Protobuf translations [DPP-104] (#8272)
* Fix typos and inconsistencies

* Implement toString for SRequiresOnLedger

* Add benchmarks

changelog_begin
changelog_end

* Add copyright headers

* Make SubmissionBuilder final

* Address missing reference ledger export on Windows

* Fix README.md

* Add clarification on README.md

* Limit visibility of reference ledger export output

* Ensure benchmark recognizes the optionality of exercise return values

* Address https://github.com/digital-asset/daml/pull/8272#discussion_r542194308

* Follow up to changes from https://github.com/digital-asset/daml/pull/8273
2020-12-15 11:49:52 +00:00
Samir Talwar
7375afd4f3
Release 1.8.0-snapshot.20201214.5841.0.a8ae8e4a. (#8290)
This is not intended for general consumption.

This reverts the following PRs, in reverse order:

- #8082
- #8146
- #8153
- #8180
- #8194

CHANGELOG_BEGIN
- [Integration Kit] Reverted a fix to the kvutils input state that
  caused performance degradation on some ledgers. This is a temporary
  measure that we do not expect to be used outside of very specific
  circumstances, as it can trade correctness for performance if the
  semantics are not completely understood.
CHANGELOG_END
2020-12-15 10:57:54 +00:00
Moritz Kiefer
7d6e8f8d5c
Do not checkout release commit in scala 2.13 (#8291)
* Do not checkout release commit in scala 2.13

At least for now, this fails because older commits don’t have the
necessary infrastructure so far. At some point we probably do want to
run this on release commits but that can wait a bit and this unblocks
release PRs for bugfix releases.

changelog_begin
changelog_end

* Update azure-pipelines.yml

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-12-15 10:13:24 +00:00
Kamil Bożek
ad432592b8
Multi-party submissions in kvutils [KVL-707] (#8275)
* Multi-party submission authorization in the TransactionCommiter

* Unit tests for committer's authorization

* Reduced code duplication in TransactionCommitterSpec

* Formatted changes

* Changelog
CHANGELOG_BEGIN
CHANGELOG_END

* Formatting fixes

* Fixed a typo

* Formatted changes
2020-12-15 10:14:18 +01:00
Remy
91b53dd3f1
LF: Archive decoder reject choice observers for LF < 1.dev (#8283)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-15 09:42:54 +01:00
Samir Talwar
70af09db51
kvutils: Generalize caching. [KVL-747] (#8262)
* kvutils: Remove QueryableReadSet, and simplify caching a little.

* kvutils: Generalize the caching ledger state reader.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Make StateReader even more general (no Option).

* kvutils: Convert LedgerStateReaderWithFingerprints to a type alias.

And DamlLedgerStateReaderWithFingerprints too.

* kvutils: Replace LedgerStateReaderWithFingerprintsFromValues.

With LedgerStateOperationsReader and a `.map`.

* kvutils: Replace RawToDamlLedgerStateReaderWithFingerprintsAdapter.

With a `.comapKeys`.

* kvutils: Delete duplicate caching code.

* caching: Move `Cache#get` into its own subtrait.

This method is used by the submission validator, but not by the
caching state readers.

* caching: Move cache mapping from CachingStateReader into Cache.

* caching: Remove an over-eager comment.

* kvutils: Rename `StateReader#comapKeys` to `contramapKeys`.

I got contravariant functors confused with cofunctors.

* caching: Add Scaladoc and improve parameter names in MappedCache.

* kvutils: Adapters everywhere!

* caching: Rename the base test class.
2020-12-14 20:13:34 +00:00
Moritz Kiefer
1b32044a74
Build //libs-scala/... on 2.13 (#8282)
* Build //libs-scala/... on 2.13

One test is unfortunately disabled at the moment since I utterly
failed to figure out why I get a ClassNotFoundException on 2.13.

changelog_begin
changelog_end

* Copyright headers

changelog_begin
changelog_end

* I can’t bazel today

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Update libs-scala/resources/src/main/2.13/com/daml/resources/UnitCanBuildFrom.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* No split on view

changelog_begin
changelog_end

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-12-14 17:36:53 +00:00
Remy
9bc2e4bd2c
LF: fix some of the choice-observer FIXMEs (#8281)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-14 18:26:49 +01:00
Sofia Faro
e52c2a0252
Bundle ghcversion.h and hpp with damlc. (#8278)
* Bundle hpp with damlc.

This PR eliminates the red squiggly lines you get when opening up the
standard library in daml studio (e.g. after a Go To Definitien).

(When CPP is used, Go To Definition is still pointing to the wrong place,
because the line numbers are messed up. But it's better than having a
million error messages.)

We could also remove --cpp flag in a separate PR, though that may be
considered a breaking change.

changelog_begin
changelog_end

* cleanup locateRunfiles jank

* Revert "cleanup locateRunfiles jank"

This reverts commit 82552003ae.
2020-12-14 17:23:41 +00:00
Remy
7d31135645
LF: retire VersionTimeline (#8273)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-14 16:56:47 +01:00
Robin Krom
54e442b2e5
damlc: warn on bad package names/versions. (#8254)
* damlc: warn on bad package names/versions.

This fixes #7208 and #7317. When package names or versions are
encountered in the daml.yaml that will be rejected by ghc-pkg when the
package is loaded as dependency, a big warning is output.

CHANGELOG_BEGIN
CHANGELOG_END

* hlint

* multiline strings

* update version regex

* update package name regex

* better error formatting regex printing
2020-12-14 16:44:43 +01:00
Moritz Kiefer
0d27c33da3
Disable create-daml-app-tests with default project name (#8277)
These are basically identical to create-daml-app-tests-proj-name
except for the fact that they use the default project name and they
are super slow so we don’t want to run them twice.

changelog_begin
changelog_end
2020-12-14 14:25:17 +01:00
Moritz Kiefer
d24ef94f36
Add a Scala 2.13 build pipeline (#8271)
* Add a Scala 2.13 build pipeline

This adds initial support for multiple Scala versions controlled via
the DAML_SCALA_VERSION env var and a CI job to make sure we don’t
regress. For now we only test //libs-scala/ports/... which seemed like
the easiest starting point I could find. We can incrementally expand
that over time.

changelog_begin
changelog_end

* Document pinning

changelog_begin
changelog_end

* Address review comments

changelog_begin
changelog_end
2020-12-14 12:42:26 +01:00
tudor-da
ae28cf40c2
Safely update ledger end (#8221)
CHANGELOG_BEGIN
[Integration Kit] Re-enabled asynchronous commits in JdbcIndexer.
CHANGELOG_END
2020-12-14 12:07:03 +02:00
Moritz Kiefer
76b6fd86fb
Upgrade Scala dependencies for 2.13 compatibility (#8268)
* Upgrade Scala dependencies for 2.13 compatibility

This upgrades a bunch of Scala libraries to versions that have 2.13
support. There are two libraries that are still missing:

- diffson, this has a new version but with significant breaking
  changes and it is only used in Naigator console which I hope to kill
  before I have to worry about this.

- ai.x:diff, this is used in the ledger API test tool. The library is
  abondened but there are a few alternatives.

changelog_begin
changelog_end

* Fix pureconfig

changelog_begin
changelog_end

* Fix Navigator

changelog_begin
changelog_end
2020-12-14 09:59:00 +01:00
Remy
9dbc6dbfcc
LF: use constants for language/transaction versions. (#8261)
We use the constants defined in LanguageVersion/TransactionVersions instead 
repeating the somewhat magic constants "6", "7", "8", "10" and "dev" in multiple places?

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 19:31:46 +01:00
Gary Verhaegen
acf654ca55
replace new Prim with Prim.valueOf (#8270)
Creating new primitives has always been a bad idea.

Stolen from #8267 which stole it from #975.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 18:50:37 +01:00
Remy
3783a158ff
Bump perf test (#8269)
Needs bumping due to the API changes for GenNode (#8217)

changelog_begin
changelog_end
2020-12-11 17:37:44 +00:00
Samir Talwar
c15071e8ca
kvutils: Extract out a common type for the ledger state readers. (#8264)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:59:47 +00:00
Remy
44c5b8a777
LF: remove version from value nested in GenNode. (#8217)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:12:14 +01:00
Gary Verhaegen
73724bccb8
protect trigger migrations (#8265)
This prevents the `dade-copyright-headers` script from changing those
files (and from checking them for copyright headers, unfortunately).

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:05:37 +01:00
Gary Verhaegen
ffc3044b39
more idempotent copyright updater (#8263)
At the moment, updating copyright headers requires the following dance:

1. Update the COPY file.
2. Run `./fmt.sh`
3. Update the `dev-env/bin/dade-copyright-headers` file to update the
   expected copyright marker.

Forgetting to do 3 results in `./fmt.sh` (really the underlying
`dev-env/bin/dade-copyright-headers update`) to not be idempotent, and
to add an extra 3 lines of copyright headers on each run (with
`dade-copyright-headers check`, and therefore `./fmt.sh --test`, never
succeeding). This small tweak removes step 3.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:52:06 +01:00
Remy
009b030463
LF: Simplify Language Version representation (#8258)
We drop the distinction (at the type level) of Dev and Stable language
version.  The two main reason that motivate this choice:
* we never really used this distinction.
* we want to add the concept of "preview" version (which is neither Dev nor Stable)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:21:45 +01:00
azure-pipelines[bot]
7dbb3a60ca
update NOTICES file (#8260)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-11 15:05:05 +00:00
Brian Healey
6a1e0a633b
Avoid haskell and jvm bazel blackduck scan stomping (#8247)
* Run bazel scan for all but haskell first, then haskell at end so they do not stomp on each other

CHANGELOG_BEGIN
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>

* use common prefix between runs so results are aggregated, get branch name from running job rather than assuming master

* DO NOT MERGE: disable all jobs except for blackduck scan

* haskell before full scan

* parens instead of braces

* differentiate haskell and jvm bazel runs with unique code location

* unique code location prefix

* fix syntax

* unique code location to avoid clashing bazel runs

* Use master security-blackduck script helper

* reenable all jobs to make mergeable

* cleanup whitespace

* Use Build.SourceBranchName for branch

* Update ci/cron/daily-compat.yml

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* DO NOT MERGE: skip all jobs but blackduck, skip update notices file step

* reenable all jobs to make mergeable

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-12-11 07:56:05 -05:00
Moritz Kiefer
75d28d1242
Bump perf test (#8256)
Needs bumping due to the minor API changes for multi-party submissions

changelog_begin
changelog_end
2020-12-11 12:29:49 +00:00
Gary Verhaegen
917ffe8c8b
daily ci: remove debug output (#8255)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 12:09:29 +00:00