Commit Graph

5736 Commits

Author SHA1 Message Date
Samir Talwar
68459e60e9
Fix the paths of the files in protobufs.zip. (#8088)
Turns out the `package_dir` is not what we're looking for. Omitting it
fixes the path.

_Before:_

```
     1766  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_8/com/digitalasset/daml_lf_1_8/daml_lf.proto
```

_After:_

```
     1766  2010-01-01 00:00   protos-0.0.0/com/digitalasset/daml_lf_1_8/daml_lf.proto
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-27 11:51:22 +00:00
Samir Talwar
9976b4cd50
Bazel: Factor out logic around Protobuf JARs. [KVL-714] (#8084)
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.

* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.

* Bazel: Factor out Java/Scala protobuf class generation into a helper.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/archive: Use `proto_jars`.

* Bazel: Remove the visibility modifier from `proto_jars`.

It's too confusing. Just make everything public.

* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.

* Bazel: Add comments to the various parts of `proto_jars`.

* daml-assistant: Do unpleasant things with `location` in Bazel.
2020-11-27 08:34:53 +00:00
Remy
2b67c74d9a
LF: clean scala Decoder test. (#8077)
As requested by Martin in #8069, we extract most of the changes from
DecodeV1Spec in a separate PR.
This advances the state of #7155

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 16:16:27 +01:00
Remy
174ba1de30
Compiler: Clean compiler from from Exercise's actors (#8076)
This is the Haskell counterpart of #8071.
This advances the state of #7155.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 15:38:30 +01:00
Richard Kapolnai
de498e4b0c
Add party management service to the services page, targeting #8027. (#8075)
* Add party management service to the services page, targeting #8027.
CHANGELOG_BEGIN
CHANGELOG_END

* rephrase to make it more compact

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* remove unnecessary note

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* remove unnecessary note

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* remove blank line

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2020-11-26 14:51:08 +01:00
Oliver Seeliger
8cfe6c3744
Remove ParticipantPruningIT from compatibility exceptions (#8072)
As ParticipantPruningIT is not included by default on any ledger.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 14:41:19 +01:00
Remy
41543ad161
LF: drop support for Exercise's actors (#8071)
This is a first set toward dropping support of deprecated LF version
(#7155).

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 14:38:16 +01:00
tudor-da
0447910aa3
Increase precision for benchmarking results in integrity-checker (#8054)
* Increase precision for benchmarking results in integrity-checker

CHANGELOG_BEGIN
CHANGELOG_END

* Update ledger/participant-state/kvutils/tools/src/main/scala/ledger/participant/state/kvutils/tools/integritycheck/IntegrityChecker.scala

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-11-26 14:21:16 +01:00
Gary Verhaegen
8dba45a478
minor docs edits (#8073)
A question on discuss prompted me to read this page, and I spotted a
couple typos.

As a side note, I find references to DAML-LF versions unhelpful. I don't
think it's fair to expect our users to know the mapping between DAML
versions and DAML-LF versions.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 10:57:53 +00:00
Samir Talwar
47b68bc554
participant-integration-api: Add unit tests for LedgerConfigProvider. (#8057)
* Use `WriteConfigService` in LedgerConfigProvider.

* Write unit tests for LedgerConfigProvider.

CHANGELOG_BEGIN
CHANGELOG_END

* Use explicit configurations in LedgerConfigProviderSpec.
2020-11-26 10:03:29 +00:00
Samir Talwar
7948ca36c8
Bazel: Document attaching the Java sources in IntelliJ. (#8070)
CHANGELOG_BEGIN
CHANGELOG_END
2020-11-26 09:33:18 +00:00
Robert Autenrieth
8bec837560
Support multiple submitters in kvutils protobuf [KVL-704] (#8043)
* Support multiple submitters in kvutils protobuf

CHANGELOG_BEGIN
CHANGELOG_END

* Sort submitters for the deduplication key

* Apply suggestions (style)

* Add test for command deduplication keys
2020-11-26 09:45:49 +01:00
Gary Verhaegen
3cef53135c
ci/cron: do not push artifacts to gcs bucket (#8067)
Having the cron push artifacts to GCP was really only meant to happen
once. I got distracted and worked on other things. This PR closes that
work loop such that the current state and expectations are:

- Every new release pushes to GCP as part of the release process.
- The cron only checks that the GCP backup exists and matches, but does
  not push if it doesn't.

The reason for this is we want the cron job to fail if there are
additional, unexpected files in a release, rather than automatically
commit those files for the long term.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-25 19:12:03 +01:00
Moritz Kiefer
06bec6a591
Upgrade highlight.js (#8065)
Addresses security advisory. I upgraded typedoc as well since that
pulls in the older version of highlight.js and I dont like breaking bounds.

changelog_begin
changelog_end
2020-11-25 17:23:09 +01:00
Richard Kapolnai
239f4df945
Link concepts "gRPC Ledger API" and "Ledger API" in glossary for #8029. (#8051)
* Link concepts "gRPC Ledger API" and "Ledger API" in glossary for #8029.
CHANGELOG_BEGIN
CHANGELOG_END

* remove gRPC Ledger API which was in the wrong place

* rephrase alternative names

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2020-11-25 16:28:33 +01:00
nickchapman-da
654f5e33b6
Fix tail-calls to execute in constant space by doing early pop of temporaries stack. (#8036)
changelog_begin
changelog_end
2020-11-25 15:12:21 +00:00
Moritz Kiefer
4e39e6163b
Get lsof from dev-env (#8064)
On MacOS we don’t seem to have lsof by default which disables our
sandbox cleanup for stray instances.

changelog_begin
changelog_end
2020-11-25 14:43:37 +00:00
Gary Verhaegen
b23304c691
add default capability to macos (#5915)
This is the macOS part of #5912, which I have separated because our
macOS nodes have a different deployment process so it seemed easier to
track the deployment of the change separately.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-25 15:34:33 +01:00
Robin Krom
a0ccfdb80e
docs: improve upgrade docs (#8047)
This addresses feedback on the upgrade documentation. We switch the
example from coins to carbon certificates and add a section for the
centralized database case to make the base case clear.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-25 15:24:57 +01:00
Kamil Bożek
ef6fdc2306
Certificate revocation checks using the OCSP [KVL-713] (#7965)
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.

The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.

CHANGELOG_BEGIN

- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature

CHANGELOG_END
2020-11-25 14:28:32 +01:00
azure-pipelines[bot]
2094786ac0
rotate release duty after 1.8.0-snapshot.20201124.5709.0.dabd55d0 (#8056)
@cocreature is taking care of 1.8.0-snapshot.20201124.5709.0.dabd55d0 (#8055), so they get pushed back to the end of the line.

Please do not merge this before #8055.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-11-25 13:44:32 +01:00
Moritz Kiefer
2d3820ac14
Upgrade akka-http to 10.2 (#8058)
* Upgrade akka-http to 10.2

Follow up to #8048, I left out this upgrade to reduce noise and since
I wasn’t quite sure how involved it was going to be.

changelog_begin
changelog_end

* Reenable transparent HEAD requests

Apparently no longer on by default but we depend on this in waitForHttpServer

changelog_begin
changelog_end
2020-11-25 13:39:25 +01:00
Andreas Lochbihler
2156f946d1
kvutils reports lookupByKey inconsistencies as Inconsistent (#7914)
CHANGELOG_BEGIN
kvutils reports LookupByKey node mismatches during validation as Inconsistent
instead of Disputed if they can be due to contention on the contract key
CHANGELOG_END
2020-11-25 13:36:26 +01:00
Moritz Kiefer
037ff64e96
Bump timeout in integration tests (#8059)
changelog_begin
changelog_end
2020-11-25 10:47:01 +00:00
azure-pipelines[bot]
d8a5a0e516
release 1.8.0-snapshot.20201124.5709.0.dabd55d0 (#8055)
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.

@cocreature is in charge of this release.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-11-25 10:24:35 +01:00
Moritz Kiefer
d83cbdb475
Upgrade akka and akka-http (#8048)
* Upgrade akka and akka-http

Was chasing an issue somewhere and thought this might affect it in
some way. It didn’t but I might as well turn the upgrade into a PR.

changelog_begin
changelog_end

* Fix trigger service tests

changelog_begin
changelog_end

* Downgrade akka-http again

changelog_begin
changelog_end

* Upgrade akka-http again and fix tests

changelog_begin
changelog_end

* Cleanup trigger service

changelog_begin
changelog_end
2020-11-25 10:13:51 +01:00
Miklos
6664794600
[kvutils] Add config key to read set [KVL-746] (#8052) 2020-11-25 10:07:21 +01:00
Robin Krom
dabd55d0b0
assistant integratin tests: speed and cleanup. (#8032)
CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 19:05:02 +01:00
Gary Verhaegen
1aadbb1d6f
ts: support GenMaps (#8040)
ts: support GenMaps

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 17:11:53 +01:00
Richard Kapolnai
f3caa5919d
rename "gRPC API" to "gRPC Ledger API" (#8046)
* rename "gRPC API" to "gRPC Ledger API"
an incremental step towards #8029
changelog_begin
changelog_end

* fix section underline
2020-11-24 16:34:07 +01:00
Martin Huschenbett
38a9c70a79
DAML-LF spec: Some minor fixes (#8050)
We correct some spelling mistakes and make the formatting of some rules a bit more consistent.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 15:07:54 +00:00
Remy
0e4adfc47b
LF: clean LF encoder from deprecated LF versions (#8049)
This advances the state of #7788.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 15:56:13 +01:00
Remy
68c92b479c
LF: version transaction node independently in proto (#7925)
This advances the state of #7788.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 14:07:33 +01:00
Stefano Baghino
d5ff0dd8ea
Refactor indexing to minimize transaction traversals (#8019)
* Refactor indexing to minimize transaction traversals

changelog_begin
[Integration Kit] Bugfix: daml.index.db.store_transaction metrics were keeping track of empty insertions, skewing down the numbers
[Integration Kit] Performance: minimizing the number of traversals of a transaction to index it, more efficient indexing
changelog_end

* Add empty stakeholders for non-consuming exercises

* Fix test to check usage of pre-computed blinding info

* Remove unnecessary string interpolators

* Shorten creation of net visibility

* Delete obsolete comment

* Add visibility of divulged contracts

* Address review https://github.com/digital-asset/daml/pull/8019#discussion_r527666572

* Address review https://github.com/digital-asset/daml/pull/8019#pullrequestreview-535399441

* Address review https://github.com/digital-asset/daml/pull/8019#discussion_r527681774

* Fix ordering of batches, use explicitly named types to avoid confusion

* Default to an empty set if a divulged contract is not visible in the current transaction

* Shorten references to store transaction metrics when writing

* Address review https://github.com/digital-asset/daml/pull/8019#discussion_r527745640

* Add divulgences from past transactions

* More fine-grained types and signatures
2020-11-24 13:08:35 +01:00
Samir Talwar
09d7400e22
kvutils/tools: Verify that write sets are readable. [KVL-493] (#8038)
* kvutils/tools: Factor out builds from IntegrityCheckerSpec.

* kvutils/tools: Verify that write sets are readable.

CHANGELOG_BEGIN
- [Integration Kit] The integrity checker now validates the contents of
  each write set, regardless of whether it is used subsequently.
CHANGELOG_END

* kvutils/tools: Move the write set exception outwards.

* kvutils/tools: Make the integrity checker `main` method easier to read.

* kvutils/tools: Move `WriteSetEntries` logic into CommitStrategySupport.

This makes it more flexible with regards to extended write sets.

* kvutils/tools: Use the support state key serialization strategy.
2020-11-24 10:43:22 +00:00
Richard Kapolnai
57e0b065ad
Fix typo in json-api docs (#8045)
CHANGELOG_BEGIN
CHANGELOG_END
2020-11-24 10:46:02 +01:00
Moritz Kiefer
92cb4e7a32
Bump curl on Windows (#8044)
The old url 404s now and upgrading seems nice anyway.

changelog_begin
changelog_end
2020-11-24 09:27:05 +00:00
fabiotudone-da
1bd98c068d
Address review comments in #8009 and #8012 [KVL-736] (#8024)
* Address review comments in #8009 and #8012

CHANGELOG_BEGIN
CHANGELOG_END

* Tidy up

* Make test more meaningful and precise
2020-11-24 09:17:01 +01:00
Robert Autenrieth
07adbc8d80
Improve logging related to ledger config (#8025)
* Improve logging related to ledger config

CHANGELOG_BEGIN
CHANGELOG_END

* Update ledger/participant-integration-api/src/main/scala/platform/apiserver/services/LedgerConfigProvider.scala

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Improve wording

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-11-23 21:47:03 +01:00
Oliver Seeliger
7a2a2700cf
Introduce optional ledger-api-test-tool tests with ParticipantPruningIT (#8041)
CHANGELOG_BEGIN
CHANGELOG_END
2020-11-23 19:09:39 +01:00
Oliver Seeliger
83f11aa784
Participant pruning ledger api server support ported from Canton (#7988)
* Participant pruning ledger api server support ported from Canton

CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END

* Review feedback from Stefano

* Add pruning tests plus missed command completions change

* Review feedback from Robert

* Improved test readability by having populate helper return offsets

* Review feedback

* Ledger api changes to pruning api and disable canton pruning test

- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string

* Review feedback: Use ApiOffsetSConverter for logged offsets
2020-11-23 15:28:04 +01:00
Robin Krom
936a0862fb
daml codegen: Add deprecation warning for scala. (#8033)
This fixes #8022. This adds a deprecation warning for the scala codegen.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-23 13:27:49 +01:00
tudor-da
f7c2ca808a
Allow async commits only within JdbcIndexer (#8004)
* Allow async commits within JdbcIndexer (#8004)

CHANGELOG_BEGIN
CHANGELOG_END

* Test in SqlLedgerSpec asserting no async commits in JdbcLedgerDao
2020-11-23 11:13:23 +01:00
Miklos
d5de65a234
Support serializing a single state update [KVL-669] (#8031) 2020-11-20 18:04:25 +01:00
Samir Talwar
bd037475eb
kvutils/tools: Fix the integrity checker so it compares state updates. (#8030)
* kvutils/tools: Fix the integrity checker so it compares state updates.

Previously, it would always take the list of updates _before_ processing
the import, leading to this useless check:

    Comparing expected and actual state updates.
    Successfully compared 0 state updates.

This makes the snapshotting lazy (by using a function instead of a
value), ensuring we actually perform the assertion.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils/tools: Add a test for the state updates regression.
2020-11-20 15:46:42 +00:00
Moritz Kiefer
65f355f327
Wait for live event in build-and-lint JS tests (#8001)
The stream from the JSON API is split into two sections:

1. The unordered section of ACS events.
2. The ordered transaction stream after the live event.

Our assertions assume that we only get 2 but currently we don’t wait
before sending commands so depending on timing, some of the events can
be delivered as part of the ACS section in a different order causing
very confusing assertion failures.

This test fixes this by waiting for the live event indicating the end
of section 1 and thereby forcing everything to come via the ordered
transaction stream.

Verified that this fixes the flakiness with --runs_per_test=50 which
reproduced it fairly reliable before (on CI, never managed to hit it
locally).

changelog_begin
changelog_end
2020-11-20 14:38:49 +01:00
Martin Huschenbett
da551ea142
DAML-LF spec: Explain how to use LaTeX input (#8018)
Add a description of how to enable LaTeX input mode when editing the
DAML-LF spec in VS Code. Also add some additional symbols.

CHANGELOG_BEGIN
CHANGELOG_END
2020-11-20 13:41:10 +01:00
fabiotudone-da
13851ca5e9
[KV] Read and prefer pre-computed blinding info when decoding KV transaction log entries [KVL-736] (#8012)
* Test and fix Conversions#encodeBlindingInfo

* Fix rebase

* Read and prefer pre-computed blinding info when decoding KV transaction log entries

CHANGELOG_BEGIN
CHANGELOG_END

* Fix rebase

* Tidy up

* Tidy up
2020-11-20 10:41:58 +01:00
Richard Kapolnai
34d7429147
fix section link in docs (#7850)
changelog_begin
changelog_end
2020-11-20 08:42:24 +00:00
fabiotudone-da
4f21453c5a
[KV] Fill DamlTransactionEntry#bindingInfo during TransactionCommitter#blind [KVL-736] (#8009)
* Fill DamlTransactionEntry#bindingInfo during TransactionCommitter#blind

CHANGELOG_BEGIN
CHANGELOG_END

* Test and fix Conversions#encodeBlindingInfo

* Fix rebase

* Fix non-determinism

* Reduce visibility

* Tidy up
2020-11-20 08:43:08 +01:00