Commit Graph

5895 Commits

Author SHA1 Message Date
Samir Talwar
70af09db51
kvutils: Generalize caching. [KVL-747] (#8262)
* kvutils: Remove QueryableReadSet, and simplify caching a little.

* kvutils: Generalize the caching ledger state reader.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Make StateReader even more general (no Option).

* kvutils: Convert LedgerStateReaderWithFingerprints to a type alias.

And DamlLedgerStateReaderWithFingerprints too.

* kvutils: Replace LedgerStateReaderWithFingerprintsFromValues.

With LedgerStateOperationsReader and a `.map`.

* kvutils: Replace RawToDamlLedgerStateReaderWithFingerprintsAdapter.

With a `.comapKeys`.

* kvutils: Delete duplicate caching code.

* caching: Move `Cache#get` into its own subtrait.

This method is used by the submission validator, but not by the
caching state readers.

* caching: Move cache mapping from CachingStateReader into Cache.

* caching: Remove an over-eager comment.

* kvutils: Rename `StateReader#comapKeys` to `contramapKeys`.

I got contravariant functors confused with cofunctors.

* caching: Add Scaladoc and improve parameter names in MappedCache.

* kvutils: Adapters everywhere!

* caching: Rename the base test class.
2020-12-14 20:13:34 +00:00
Moritz Kiefer
1b32044a74
Build //libs-scala/... on 2.13 (#8282)
* Build //libs-scala/... on 2.13

One test is unfortunately disabled at the moment since I utterly
failed to figure out why I get a ClassNotFoundException on 2.13.

changelog_begin
changelog_end

* Copyright headers

changelog_begin
changelog_end

* I can’t bazel today

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Update libs-scala/resources/src/main/2.13/com/daml/resources/UnitCanBuildFrom.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* No split on view

changelog_begin
changelog_end

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-12-14 17:36:53 +00:00
Remy
9bc2e4bd2c
LF: fix some of the choice-observer FIXMEs (#8281)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-14 18:26:49 +01:00
Sofia Faro
e52c2a0252
Bundle ghcversion.h and hpp with damlc. (#8278)
* Bundle hpp with damlc.

This PR eliminates the red squiggly lines you get when opening up the
standard library in daml studio (e.g. after a Go To Definitien).

(When CPP is used, Go To Definition is still pointing to the wrong place,
because the line numbers are messed up. But it's better than having a
million error messages.)

We could also remove --cpp flag in a separate PR, though that may be
considered a breaking change.

changelog_begin
changelog_end

* cleanup locateRunfiles jank

* Revert "cleanup locateRunfiles jank"

This reverts commit 82552003ae.
2020-12-14 17:23:41 +00:00
Remy
7d31135645
LF: retire VersionTimeline (#8273)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-14 16:56:47 +01:00
Robin Krom
54e442b2e5
damlc: warn on bad package names/versions. (#8254)
* damlc: warn on bad package names/versions.

This fixes #7208 and #7317. When package names or versions are
encountered in the daml.yaml that will be rejected by ghc-pkg when the
package is loaded as dependency, a big warning is output.

CHANGELOG_BEGIN
CHANGELOG_END

* hlint

* multiline strings

* update version regex

* update package name regex

* better error formatting regex printing
2020-12-14 16:44:43 +01:00
Moritz Kiefer
0d27c33da3
Disable create-daml-app-tests with default project name (#8277)
These are basically identical to create-daml-app-tests-proj-name
except for the fact that they use the default project name and they
are super slow so we don’t want to run them twice.

changelog_begin
changelog_end
2020-12-14 14:25:17 +01:00
Moritz Kiefer
d24ef94f36
Add a Scala 2.13 build pipeline (#8271)
* Add a Scala 2.13 build pipeline

This adds initial support for multiple Scala versions controlled via
the DAML_SCALA_VERSION env var and a CI job to make sure we don’t
regress. For now we only test //libs-scala/ports/... which seemed like
the easiest starting point I could find. We can incrementally expand
that over time.

changelog_begin
changelog_end

* Document pinning

changelog_begin
changelog_end

* Address review comments

changelog_begin
changelog_end
2020-12-14 12:42:26 +01:00
tudor-da
ae28cf40c2
Safely update ledger end (#8221)
CHANGELOG_BEGIN
[Integration Kit] Re-enabled asynchronous commits in JdbcIndexer.
CHANGELOG_END
2020-12-14 12:07:03 +02:00
Moritz Kiefer
76b6fd86fb
Upgrade Scala dependencies for 2.13 compatibility (#8268)
* Upgrade Scala dependencies for 2.13 compatibility

This upgrades a bunch of Scala libraries to versions that have 2.13
support. There are two libraries that are still missing:

- diffson, this has a new version but with significant breaking
  changes and it is only used in Naigator console which I hope to kill
  before I have to worry about this.

- ai.x:diff, this is used in the ledger API test tool. The library is
  abondened but there are a few alternatives.

changelog_begin
changelog_end

* Fix pureconfig

changelog_begin
changelog_end

* Fix Navigator

changelog_begin
changelog_end
2020-12-14 09:59:00 +01:00
Remy
9dbc6dbfcc
LF: use constants for language/transaction versions. (#8261)
We use the constants defined in LanguageVersion/TransactionVersions instead 
repeating the somewhat magic constants "6", "7", "8", "10" and "dev" in multiple places?

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 19:31:46 +01:00
Gary Verhaegen
acf654ca55
replace new Prim with Prim.valueOf (#8270)
Creating new primitives has always been a bad idea.

Stolen from #8267 which stole it from #975.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 18:50:37 +01:00
Remy
3783a158ff
Bump perf test (#8269)
Needs bumping due to the API changes for GenNode (#8217)

changelog_begin
changelog_end
2020-12-11 17:37:44 +00:00
Samir Talwar
c15071e8ca
kvutils: Extract out a common type for the ledger state readers. (#8264)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:59:47 +00:00
Remy
44c5b8a777
LF: remove version from value nested in GenNode. (#8217)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:12:14 +01:00
Gary Verhaegen
73724bccb8
protect trigger migrations (#8265)
This prevents the `dade-copyright-headers` script from changing those
files (and from checking them for copyright headers, unfortunately).

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:05:37 +01:00
Gary Verhaegen
ffc3044b39
more idempotent copyright updater (#8263)
At the moment, updating copyright headers requires the following dance:

1. Update the COPY file.
2. Run `./fmt.sh`
3. Update the `dev-env/bin/dade-copyright-headers` file to update the
   expected copyright marker.

Forgetting to do 3 results in `./fmt.sh` (really the underlying
`dev-env/bin/dade-copyright-headers update`) to not be idempotent, and
to add an extra 3 lines of copyright headers on each run (with
`dade-copyright-headers check`, and therefore `./fmt.sh --test`, never
succeeding). This small tweak removes step 3.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:52:06 +01:00
Remy
009b030463
LF: Simplify Language Version representation (#8258)
We drop the distinction (at the type level) of Dev and Stable language
version.  The two main reason that motivate this choice:
* we never really used this distinction.
* we want to add the concept of "preview" version (which is neither Dev nor Stable)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 16:21:45 +01:00
azure-pipelines[bot]
7dbb3a60ca
update NOTICES file (#8260)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-11 15:05:05 +00:00
Brian Healey
6a1e0a633b
Avoid haskell and jvm bazel blackduck scan stomping (#8247)
* Run bazel scan for all but haskell first, then haskell at end so they do not stomp on each other

CHANGELOG_BEGIN
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>

* use common prefix between runs so results are aggregated, get branch name from running job rather than assuming master

* DO NOT MERGE: disable all jobs except for blackduck scan

* haskell before full scan

* parens instead of braces

* differentiate haskell and jvm bazel runs with unique code location

* unique code location prefix

* fix syntax

* unique code location to avoid clashing bazel runs

* Use master security-blackduck script helper

* reenable all jobs to make mergeable

* cleanup whitespace

* Use Build.SourceBranchName for branch

* Update ci/cron/daily-compat.yml

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* DO NOT MERGE: skip all jobs but blackduck, skip update notices file step

* reenable all jobs to make mergeable

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-12-11 07:56:05 -05:00
Moritz Kiefer
75d28d1242
Bump perf test (#8256)
Needs bumping due to the minor API changes for multi-party submissions

changelog_begin
changelog_end
2020-12-11 12:29:49 +00:00
Gary Verhaegen
917ffe8c8b
daily ci: remove debug output (#8255)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 12:09:29 +00:00
Moritz Kiefer
5c077a4550
Add multi-party submissions to DAML Script (#8240)
* Add multi-party submissions to DAML Script

changelog_begin

- [DAMl Script] Add early-access submitMulti and submitMultiMustFail
  for multi-party submissions. This is only supported in DAML Studio at
  the moment.

changelog_end

* Fix daml-lf prettyprinter

changelog_begin
changelog_end

* Cleanup visibleIn

changelog_begin
changelog_end

* Fix JsonApiIt

changelog_begin
changelog_end

* s/Set()/Set.empty

changelog_begin
changelog_end

* Cleanup token check

changelog_begin
changelog_end

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/LedgerInteraction.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Document ParticipantView

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2020-12-11 12:45:32 +01:00
Robin Krom
0d37052206
fix: Create missing directories for script output (#8196)
This fixes #8142.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 11:48:31 +01:00
Kamil Bożek
5db4ca687b
Multi-party command submission [KVL-699] (#8152)
Changes:
- replaced submitter: Party with actAs: Set[Party] and readAs: Set[Party] in com.daml.ledger.api.domain.Commands
- adapted contract and key lookups for multi-party submissions

* Changed single Commands to accept multiple submitters

* Removed submitter field from Commands

* Replaced submitters with actAs and readAs in Commands

* Moved actAs and readAs to com.daml.ledger.api.Commands

* Contract lookup tests

* Key lookup tests

CHANGELOG_BEGIN
CHANGELOG_END

* Formatted code changes

* Remove unused method

* Formatting fixes

* Simplified logical condition for active contracts visibility

* Removed unused temporary method in Commands

* Unified contract key lookup query for different type of db types

* Simplified ContractReader class structure

* Improved dao unit tests

* Unit tests for lookups as a divulgee

* Reduced code duplication in unit tests

* Minor improvement

* Formatted code

* Fixed a ContractsReader's query

* Improved unit tests

* Explanation for engine's command authorizers and contract readers

* Minor fixes

* Adapted SpannedIndexService for the multi-party submission
2020-12-11 10:04:15 +01:00
dependabot[bot]
8decc31813
Bump ini from 1.3.5 to 1.3.7 (#8253)
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.7.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.7)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-12-11 10:02:07 +01:00
Andreas Herrmann
d3b4042ade
Factor out the middleware client from the trigger server (#8244)
* Factor out authorization middleware client

changelog_begin
changelog_end

* Factor out token refresh

* Factor out auth request

* Factor out auth middleware URIs

* factor out auth directive

* Factor out login directive

* Add login response type to api library

* Allow error handling in login callback

* Factor error handling out of authorize directive

* Move tagged token types into middleware api

* Factor out the auth middleware client

* Dedicated exception types in middleware client

* Handle auth middleware client exception

Restores the behavior of authorize before factoring out the client

* expose middleware URIs

* Use the middleware client in the middleware tests

* Use localhost/CALLBACK in testing

https://github.com/digital-asset/daml/pull/8244#discussion_r540328001

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-10 17:26:06 +00:00
Gary Verhaegen
77a9592936
remove navigator licenses (#8246)
This looks like completely dead code, and its purpose is arguably better
fulfilled by the NOTICES file now.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 17:31:20 +01:00
Remy
76c4380fef
LF: drop support from deprecated LF value versions. (#8083) (#8237)
* LF: drop support from deprecated LF value versions. (#8083)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 17:14:36 +01:00
Gary Verhaegen
5c8ac44049
update macOS nodes README (#8243)
This is far from perfect but removes the blatantly wrong sections of the
README.

Note: as a README change, this is not really a standard change, but
because the README is under the infra folder, this PR does need the tag
to pass CI.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 16:48:12 +01:00
Gary Verhaegen
05b5473a76
replace daml-licenses with NOTICES (#8241)
At some point in time, `daml-licenses` was supposed to list the licenses
for all of our Haskell dependencies. It's not been updated in ages, and
we now have a reliable way to get at all our dependencies.

This is arguably a bit weird since the command is `daml damlc license`,
and it lists licenses for everything (`daml license` would make more
sense), but it seems a lot better to give a correst superset than an
incorrect list.

Perhaps we can move the command to `daml license` at some point, but I'd
see that as a separate step. (That would technically be a breaking
change, but I think this one might be OK.)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 16:46:13 +01:00
Gary Verhaegen
2de96b79a6
cleanup oss-compliance & NOTICES.md (#8242)
Mostly outdated/redundant information across those; hopefully this is
now more accurate.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 16:45:56 +01:00
Moritz Kiefer
4478d63c68
Improve error messages on use of contract ids in contract keys (#8236)
* Improve error messages on use of contract ids in contract keys

We got some feedback that "Unexpected contract id" sounds like an
internal error whereas this is really a user error.

changelog_begin
changelog_end

* Include templateId in error message

changelog_begin
changelog_end
2020-12-10 16:26:55 +01:00
Gary Verhaegen
be2ad76897
fix link to AST (#8238)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 15:48:58 +01:00
curiousleo-da
f4fa88c977
Trace index DB -> Ledger API read path [LPF-223] (#8233)
CHANGELOG_BEGIN
- [Integration Kit] Emit OpenTelemetry events for transactions streamed
  to the client from the Ledger API
CHANGELOG_END
2020-12-10 15:06:17 +01:00
Gary Verhaegen
e7b3ac39b5
delete merge-prohibition (#8235)
Dead file, part of the defunct "red master" system.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 14:37:47 +01:00
Gary Verhaegen
a564a97444
pom template: https link (#8234)
More security is better.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 14:37:33 +01:00
Oliver Seeliger
e23f44f47d
Ledger api server index h2 avoid reliance on PUBLIC schema (#8223)
by replacing entire `participant_command_completions` table to
prevent assumption that tables reside in the `PUBLIC` schema
which they are not in canton. This is follow-up to #8035

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 10:11:47 +01:00
Gary Verhaegen
029c655adc
blackduck: open PR on NOTICES file change (#8215)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-10 10:08:28 +01:00
Remy
e2e0523b78
LF: fix versioning output in LF Repl. (#8219)
With this PR, the Repl now:
* outputs the transaction version (not sure why it was not
there before)
* outputs the node version, because each node is now versioned
independently (See #7788)
* drops the value version, because it is dictated by the node version
(See #7788)

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-09 17:40:12 +01:00
Gerolf Seitz
ea84b975dd
daml-sdk-head: don't install to path (#8220)
When running daml-sdk-head from circleci without set-path=no, the
installation fails because the assistant tries to get a Yes/No answer
from stdin (which is not available).

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-09 15:39:16 +00:00
Samir Talwar
79a91669b2
kvutils: Use mockito-scala's ArgumentMatchersSugar. (#8218)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-09 15:34:38 +00:00
Moritz Kiefer
5f1d5ad118
Fix prettyprinting in DAML REPL (#8214)
We used a weird mix of prettyprinting and string concatenation. This
breaks as soon as you have a line break somewhere because indentation
is messed up at that point. This PR fixes that by consistently (at
least more consistently than before) using the prettyprinting lib.

fixes #8213

changelog_begin

- [DAML REPL] Fix a bug where bindings with very long types sometimes
  resulted in parse errors on following lines. See #8213

changelog_end
2020-12-09 15:24:43 +01:00
Samir Talwar
4e2e39f871
kvutils: Add a test case for conflicts during fetching by key. [KVL-759] (#8204)
* kvutils: Verify the engine handles an out-of-date pre-execution.

* kvutils: Add a test case for conflicts during fetching by key.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: In KVTest, use `Reader` instead of `State` where possible.

* kvutils: Pull out helper methods for pre-execution tests.

* kvutils: Use `Inside` to simplify checks in KVUtilsTransactionSpec.
2020-12-09 14:01:07 +00:00
Ognjen Maric
b535c33e4d
Document the tx service behavior for filters and transient contracts (#8016)
Document the tx service behavior for filters and transient contracts

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-09 14:48:39 +01:00
Moritz Kiefer
aa3caec47f
Bump rules_nodejs (#8211)
* Bump rules_nodejs

rules_nodejs finally added support for module remapping via js_library
so we can kill our custom rule for that.

changelog_begin
changelog_end

* Fix windows

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2020-12-09 10:45:35 +00:00
azure-pipelines[bot]
584cdd49d9
rotate release duty after 1.8.0-snapshot.20201209.5848.0.e27026ee (#8210)
@S11001001 is taking care of 1.8.0-snapshot.20201209.5848.0.e27026ee (#8209), so they get pushed back to the end of the line.

Please do not merge this before #8209.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-09 08:01:15 +01:00
azure-pipelines[bot]
e27026ee5d
update compat versions for 1.8.0-snapshot.20201208.5840.0.38455e8c (#8206)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-09 06:41:45 +01:00
Moritz Kiefer
ec0fcb39f1
Fix docs cron again (#8208)
Unfortunately, I missed the fact that we had our own logic for
handling process failures which resulted in uncatchable
exceptions. I’ve changed one place to use the upstream handling and
the other to call `fail` which throws an IOException like I would have
expected.

changelog_begin
changelog_end
2020-12-08 22:06:22 +01:00
Moritz Kiefer
0c7791bc1c
Fix docs cron (#8207)
The change in #8191 to publish daml on sql docs failed because the
versions.json and snapshots.json files don’t exist initially. This PR
fixes that by catching the exception and treating it as an empty file.

changelog_begin
changelog_end
2020-12-08 21:32:33 +01:00