Commit Graph

7849 Commits

Author SHA1 Message Date
Moritz Kiefer
9b0fa29aec
Separate exercise & fetch for interfaces from templates (#10908)
* Separate exercise & fetch for interfaces from templates

part of #10810

changelog_begin
changelog_end

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2021-09-16 12:05:11 +00:00
nicu-da
f4adee91ca
Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
* Add conformance test for command deduplication using the CommandService

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-16 11:16:39 +00:00
Gerolf Seitz
8a391189a6
Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
* Rename Completion.deduplication_time to deduplication_duration

CHANGELOG_BEGIN
CHANGELOG_END

* Breaking protobuf change: regenerate `buf` image

Breaking-Proto: true

* Re-compute sha256s of migrations

* Fix expected error message

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-16 10:41:45 +00:00
Moritz Kiefer
8e22bb6b2d
Drop ContractId typeparameter from Value (#10827)
99% of our usecases use Value[ContractId] so this PR just fixes it.

The few other usescases are:

1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.

We don’t have any code which benefits from being polymorphic in the
contract id type.

changelog_begin
changelog_end
2021-09-16 08:46:57 +00:00
Moritz Kiefer
45437059d4
Drop cocreature as a codeowner from runtime things (#10906)
Just cleaning up my notifications a bit

changelog_begin
changelog_end
2021-09-16 09:38:12 +02:00
Moritz Kiefer
e6e81470bb
Release 1.17.0 RC from second to last commit (#10904)
changelog_begin
changelog_end
2021-09-16 09:10:54 +02:00
Moritz Kiefer
f08ac5f65c
Desugar interface implements declarations (#10895)
* Desugar interface implements declarations

This PR adds desugaring for tplImplements. This consists of the
corresponding typeclass instance (which we just ignore in LF for now,
we probably need it once we have pure functions) and a _implements_
top-level value.

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2021-09-15 19:48:59 +00:00
Hubert Slojewski
b5648c0e3d
Make CommandTracker distinguish submissions of the same command using submissionId [KVL-1104] (#10868)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 14:15:10 +00:00
nicu-da
b4328b3dc3
ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication

CHANGELOG_BEGIN

CHANGELOG_END

* Fix broken test and use consistency for tests

* ledger-api-test-tool - Add conformance test for parallel command deduplication

CHANGELOG_BEGIN
CHANGELOG_END

* Add import for 2.12 compat

* Add silencer plugin

* Split parallel command deduplication scenario into it's own test suite

* Add the parallel command deduplication test to append only ledgers

* Run parallel command deduplication tests for append only ledgers

* Apply suggestions from code review

Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>

* Code review renames

* Add compat import

* Run the test concurrently
2021-09-15 12:15:13 +00:00
Marton Nagy
0c32e3baff
Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing

* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 11:12:17 +00:00
Oliver Seeliger
b3e4975795
Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness

CHANGELOG_BEGIN
CHANGELOG_END

* Make schema migration retries and backoff configurable

* Review feedback - use RetryStrategy.constant instead

* Remove unused tailrec import

* Simplifications and runF by Marton

* Rename config options from retry to attempt and default to 30 attempts
2021-09-15 10:51:03 +00:00
nicu-da
e4cce53957
Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result

The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.

CHANGELOG_BEGIN

CHANGELOG_END

* Do not call duplicate command exception twice
2021-09-15 10:45:23 +00:00
Kamil Bozek
a939594025
Sandbox on H2 - performance improvements for the append-only schema [DPP-600] (#10888)
* Added --database-connection-pool-size parameter to the sandbox CLI

* Improved maximum ledger time lookup query

* Improved active contract lookup query

* Improved max event sequential id query

CHANGELOG_BEGIN
- [Sandbox] - Added a CLI parameter for configuring the number of connections in the database connection pool used for serving ledger API requests
CHANGELOG_END
2021-09-15 12:21:19 +02:00
tudor-da
9a1a1015f2
Increase timeout for heavy tests in ParticipantPruningIT (#10894)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 08:52:33 +00:00
Stefano Baghino
9093c6c7b3
Improve wording for the active contracts service description (#10880)
* Improve wording for the active contracts service description

This should eliminate sources of confusion as mentioned here: https://discuss.daml.com/t/offset-is-null-while-getting-active-contracts/3118

changelog_begin
[Docs] Improved description of the purpose and usage of the active contracts service
changelog_end

* Address https://github.com/digital-asset/daml/pull/10880#pullrequestreview-754078207

* Remove sand-which.
2021-09-15 10:27:40 +02:00
Stefano Baghino
c12f546774
Document #10780 (#10781)
changelog_begin
[Docs/JSON API] documented 256B limitation of Oracle query store
changelog_end
2021-09-15 10:26:28 +02:00
azure-pipelines[bot]
5814f6ad9d
update NOTICES file (#10893)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-09-15 07:30:50 +00:00
tudor-da
38227a8ed7
[Ledger API error codes] ErrorCode enrichments [DPP-591] (#10874)
* [Daml error codes API] Further implementations
* Implements ErrorCode.asGrpcError (and test)
* Error code logging now accepts correlation id and an extra context map
* Full error context is included into enriched logging context

CHANGELOG_BEGIN
CHANGELOG_END

* Fixed Scala 2.12 compilation issues
2021-09-15 07:43:32 +02:00
Brian Healey
e7c443a596
enable json index for all fields that are queried with JSON_EXISTS (#10658)
* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against

CHANGELOG_BEGIN
CHANGELOG_END

* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against
recompute sha

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 18:30:49 -04:00
Gary Verhaegen
6c1c02aeea
document complete authorized auth0 setup (#10881)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 19:11:21 +00:00
Hubert Slojewski
e4230dc51a
Do not drop generated submissionIds in GrpcCommandService [KVL-1104] (#10882)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 17:19:26 +00:00
Gary Verhaegen
b4750a495c
trigger reach auth on internal network (#10844)
In many network setups, there will be a more direct route for the
trigger service to contact the auth middleware than going back through
the frontend's public IP address (and possibly thus also through
intermediaries like an nginx reverse proxy etc.). In _some_ network
setups, it may not even be possible for the trigger service to reach the
auth middleware through its externally-visible address.

This PR caters to these cases by allowing the trigger service to use two
separate addresses for the auth middleware, an internal one the trigger
service uses when it needs to talk to the auth middleware, and an
external one used in generating URLs for external clients.

This is backwards-compatible: if the old option is used, we simply use
the same value for both.

CHANGELOG_BEGIN
- The Trigger Service can now accept separate `--auth-internal` and
  `--auth-external` CLI arguments, where `--auth-internal` is the
  address used by the Trigger Service to reach the Auth Middleware
  directly, and `--auth-external` is the address the Trigger Service uses
  in generated URLs sent back to the client. The `--auth` option remains
  and keeps working as before, setting both internal and external
  addresses to the same given value.
CHANGELOG_END
2021-09-14 14:16:42 +00:00
Gary Verhaegen
79080839c1
add Auth0 support to create-daml-app (#10673)
As I was adding support to a sample app for the third time, I decided I
might as well clean it up a bit and add support for everyone.

This is completely optional, activated by (build-time) environment
variables, and should be trivial to remove for users who don't care
about it. In an ideal world we'd have options to `daml new` and this
would be one, but I think adding it out-of-the-box is a better
alternative than letting people figure this out on their own.

CHANGELOG_BEGIN
- The `create-daml-app` template now includes support for a third
  authentication scheme (in addition to the existing "dev mode" and Daml
  Hub support): Auth0.
CHANGELOG_END
2021-09-14 15:56:45 +02:00
Stefano Baghino
b86490c891
Add @adriaanm-da to the release rotation (#10872)
changelog_begin
changelog_end
2021-09-14 14:54:32 +02:00
Moisés Ackerman
9e918c37a3
Update trigger-service docs to use --dar option in the corresponding example (#10877)
changelog_begin
changelog_end
2021-09-14 14:46:31 +02:00
Artúr Poór
49a9556e5c
[docs] Fix minor typo in doc of exerciseByKey in TS. (#10863)
changelog_begin
changelog_end
2021-09-14 12:30:50 +00:00
Robin Krom
f7c07eaa0c
interfaces: scala protobuf encoder (#10878)
This implements the protobuf encoder of the scala side for interfaces.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 12:27:53 +00:00
fabiotudone-da
be4e06427f
Ledger API Test Tool: support --additional tests [KVL-1100] (#10829)
* Support adding tests as an hidden option

* Simplify existing suites

CHANGELOG_BEGIN
CHANGELOG_END

* Remove stale conformance suites from build.yml

* `--add` -> ``--additional`

* Re-add `--all-tests` as deprecated CLI option to be tested

* Move sandbox-classic pruning test to wall clock again

* Run KVCommandDeduplicationIT for sandbox append-only

* Tidy-up

* Also add participant pruning test to ledger-on-memory/single-participant

* Remove KVCommandDeduplicationIT on ledger-on-memory/append-only

* Run the full suite plus pruning (rather than just pruning) for ledger-on-memory with multiple participants and append-only

* Add KVCommandDeduplicationIT to ledger-on-memory append-only

* Exclude ConfigManagementServiceIT from ledger-on-memory append-only multi-participant

* Tidy-up

* Use KVCommandDeduplicationIT for sandbox-on-x too

* Fix merge

Add max dedup duration arg to all the test suites that include command dedup tests

* Make `--include` and `--additional` mutually exclusive

* Uniform formatting of multi-line strings

* Move exclusions after additions as they are applied last

* Re-disable deduplication test on sandbox with static time

* Re-disable deduplication test on sandbox-on-x
2021-09-14 14:00:30 +02:00
tudor-da
97e14de644
[Ledger API error codes] ErrorCode interfaces and generator [DPP-591] (#10836)
* [Ledger API error codes] ErrorCode interfaces and generator
* Implementation of annotation processor
* Implementation of DocItem generator
* Unit testing of the generator and error code logging

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments
2021-09-14 13:50:00 +02:00
pbatko-da
6dcdaa411c
[DPP-589] Add CLI flag to select minimum enabled TLS version (#10854)
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox

CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
2021-09-14 12:37:38 +02:00
matevarga-da
1fc58d93f1
Navigator customviews highlight and choices button, apply custom theme on the login screen (#10859)
* Custom views menu highlighting and choices button fixes.

* Consider theme on the login screen

* Consider theme on the login screen

Originally reported here: https://discuss.daml.com/t/how-to-highlight-selected-custom-filter-button-in-navigator-how-to-use-the-choice-field-in-a-custom-view

changelog_begin
- [Navigator] The currently selected custom view is now highlighted on the sidebar
changelog_end

* Use page.state.id and defined user object.
2021-09-14 11:32:04 +02:00
Miklos
6faddc90c8
Update Daml Documentation to reflect command deduplication related changes [KVL-1094] (#10852)
* Minor tweaks.

* Use deduplication period instead of deduplication time.

* Introduced change ID and consistent use of deduplication duration/period.

* Consistent use of deduplication duration/period.

* Added ALREADY_EXISTS to the described gRPC errors.

* Minor tweak.

CHANGELOG_BEGIN
CHANGELOG_END

* Emphasize that applications should not change the deduplication time upon a resubmission

* Fix definition of change ID and always refer to it

* Try to clarify "Application-specific IDs"

* Make command deduplication explanation unspecific w.r.t. direction

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-14 11:28:27 +02:00
nickchapman-da
7c29eee130
Cleanup normalize from svalue (#10873)
* better variable name

CHANGELOG_BEGIN
CHANGELOG_END

* remove disallowGenMapAtVersion check
2021-09-14 09:23:48 +00:00
nickchapman-da
053f22a1af
Convert SValue to Value, and normalize, in a single code pass. (#10828)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 08:07:26 +01:00
nicu-da
37a1cb21b3
compatibility-tests - Exclude CommandDeduplicationIT from running for existing 1.17 snapshots (#10866)
Reason: in between snapshots there was a proto change (field number change) which broke compatibility

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-14 06:58:42 +00:00
nicu-da
dfae9f600f
Command deduplication - better support for different deduplication modes in conformance tests [KVL-1099] (#10864)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication

CHANGELOG_BEGIN

CHANGELOG_END

* Fix broken test and use consistency for tests
2021-09-13 19:05:50 +00:00
Gary Verhaegen
6f151e287e
save kibana exports (#10861)
As explained in #10853, we recently lost our ES cluster. While I'm not
planning on trusting Google's "rolling restart" feature ever again, we
can't exclude the possibility of future similar outages (without a
significant investment in the cluster, which I don't think we want to
do).

Losing the cluster is not a huge issue as we can always reingest the
data. Worst case we lose visibility for a few days. At least, as far as
the bazel logs are concerned.

Losing the Kibana data is a lot more annoying, as that is not derived
data and thus cannot be reingested. This PR aims to add a backup
mechanism for our Kibana configuration.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 18:28:11 +00:00
akshayshirahatti-da
99f0362aac
[JSON-API] drop package token doc changes (#10865)
* Update daml documentation to reflect the deprecation of the package/internal access token

CHANGELOG_BEGIN
CHANGELOG_END

* drop the whole internal access token section
2021-09-13 16:53:19 +00:00
Hubert Slojewski
b50bb8e437
Populate definite_answer in ApiException [KVL-1004] (#10832)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 16:28:39 +00:00
Remy
a471225fa3
LF: Add missing collision check for type synonyms (#10841)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 17:52:26 +02:00
Remy
1e1c452b36
LF: drop ad-hoc FrontStack builders (#10839)
Following #10763, we drop the ad-hoc builders for `FrontStack`.

* Building a `Fronstack` from individuals elements should be done with
  standard scala buidler.

* Building a `Fronstack` from a `TraversableOne` should be done with
  the scala 2.13 `.to(FrontStack)` methd

* Building a `Fronstack` from a `ImmArray` should be done with the
  `toImmArray` method.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 17:49:09 +02:00
Robin Krom
8f5b4fa84c
interfaces: protobuf encoder haskell side (#10850)
* interfaces: protobuf encoder haskell side

This is the implementation of the protobuf encoder on the haskell side.

CHANGELOG_BEGIN
CHANGELOG_END

* missing decoder cases for added interface data constructor
2021-09-13 14:36:14 +00:00
Oliver Seeliger
63f6678d6f
ParticipantPruningIT divulgence test fixes to avoid flakiness on canton (#10860)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 14:02:21 +00:00
nicu-da
8a9d19a99b
Command deduplication - KV conformance test for usage of max deduplication duration [KVL-1098] (#10846)
* Add conformance test for KV committer usage of max deduplication duration as deduplication duration

CHANGELOG_BEGIN

CHANGELOG_END

* Run AppendOnlyKVCommandDeduplicationIT for KV ledgers that use the append-only schema
2021-09-13 06:48:31 -07:00
Remy
24fff88992
LF: Simplify TransactionBuilder (#10753)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 15:37:46 +02:00
Moritz Kiefer
9a4c9df06a
Implement LF desugaring of interface definitions (#10834)
This only handles the interface definition, not the implementation in
the template. There are also a few rough edges:

1. It maks all choices as consuming.
2. it ignores locations

But for a poc that doesn’t seem too bad.

The tests don’t do anything super useful since the typechecker falls
over but I checked tha tthe generated LF looks more or less reasonable.

changelog_begin
changelog_end
2021-09-13 15:15:31 +02:00
Robin Krom
2aaf601912
interfaces: protobuf decoder haskell side (#10849)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 14:58:48 +02:00
Robin Krom
6dc769b7ad
interfaces: lf typechecker implementation (#10843)
* interfaces: lf typechecker implementation

CHANGELOG_BEGIN
CHANGELOG_END

* review suggestions
2021-09-13 14:58:33 +02:00
Hubert Slojewski
d9178d2ac2
Clarify version usage in test tool exclusion docs (#10858)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-13 11:30:00 +00:00
Moritz Kiefer
c113954a35
Clarify docs for test tool exclusions (#10855)
* Clarify docs for test tool exclusions

changelog_begin
changelog_end

* clarify 0.0.0

changelog_begin
changelog_end
2021-09-13 10:12:20 +00:00