Commit Graph

8075 Commits

Author SHA1 Message Date
Oliver Seeliger
054c6abb63
Upgrade canton to a more recent version (#10944)
changelog_begin
changelog_end
2021-09-20 17:42:37 +00:00
akshayshirahatti-da
b8533d50e9
[JSON-API] Production/HA documentation. (#10903)
* Initial draft documentation for JSON API production recommendations

CHANGELOG_BEGIN
CHANGELOG_END

* changes based on draft review

* changes based on code-review

* more changes based on review comments

* add note on performance impact for rebuilding query store
2021-09-20 17:41:18 +00:00
Hubert Slojewski
a33176265a
Clarify CommandTracker [KVL-1104] (#10943)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 17:25:26 +00:00
akshayshirahatti-da
5244643d16
Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 17:38:53 +01:00
azure-pipelines[bot]
6cc42ee9f4
rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
@cocreature is taking care of 1.17.0-snapshot.20210915.7832.0.38227a8e (#10891), so they get pushed back to the end of the line.

Please do not merge this before #10891.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-09-20 18:30:01 +02:00
Samir Talwar
88ef05e557
sandbox-classic: Only allow --max-parallel-submissions here. (#10941)
This option is only used by Sandbox Classic and Daml Driver for SQL.
There is no reason for it to be part of the command service
configuration.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 15:38:20 +00:00
Remy
ac02dbdeb9
LF: Exhaustive test for valueTranslator. (#10927)
* LF: Exhaustive test for value translator.

CHANGELOG_BEGIN
CHANGELOG_END

* cosmetic
2021-09-20 14:21:45 +00:00
Sofia Faro
409c0b4f60
interfaces: Add to/from_interface in proto (#10937)
* interfaces: Add to/from_interface in proto

Part of #10810

changelog_begin
changelog_end

* fix typo
2021-09-20 13:44:00 +00:00
Robert Autenrieth
02c8a9dfb8
Split CommonStorageBackend (#10871)
changelog_begin
changelog_end
2021-09-20 15:22:49 +02:00
azure-pipelines[bot]
dc71a6aea8
update NOTICES file (#10932)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-09-20 14:30:48 +02:00
Hubert Slojewski
0ba54a4bf9
Add a missing test case to the CommandTrackerFlowTest (#10939)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 14:25:42 +02:00
pbatko-da
855ecdf1a7
[DPP-572] Add ledger API test case for verifying conformance to --min-tls-version flag. (#10898)
Add CLI flag to select minimum enabled TLS version for participant server

CHANGELOG_BEGIN
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
CHANGELOG_END
2021-09-20 12:17:00 +02:00
Moritz Kiefer
3e13e3d87e
Switch to stable urls in scoop python manifest (#10933)
The file changed in the master branch which broke our builds :(

changelog_begin
changelog_end
2021-09-20 12:06:58 +02:00
Hubert Slojewski
fc2c87d950
ledger-api-client: Generate a submission id if it's empty in the CommandClient [KVL-1104] (#10926)
* Throw an exception if the submission id is empty in `CommandTracker`

CHANGELOG_BEGIN
CHANGELOG_END

* Generate a submission ID in the `CommandClient` if it's empty

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 10:00:57 +02:00
mziolekda
04d8f753b2
Clean up sandbox-on-x conformance tests. (#10766)
* Clean up sandbox-on-x conformance tests.

CHANGELOG_BEGIN
CHANGELOG_END

* fix a flaky test in config management
2021-09-20 07:52:46 +00:00
Oliver Seeliger
b4541b5a4d
Logging delay of submitted commands (#10912)
* Logging delay of submitted commands

This would have helped track down the timezone issue more quickly.

CHANGELOG_BEGIN
CHANGELOG_END

* Review feedback
2021-09-20 06:43:22 +00:00
nicu-da
b8e21d8f48
Fix takeFilter for the test StreamConsumer (#10918)
* Reorder the observers for StreamConsumer to first filter and then take just one element, not the other way around

CHANGELOG_BEGIN
CHANGELOG_END

* Refactor the test to get better stacktraces during failures

* Remove unused import
2021-09-17 16:16:04 +02:00
Remy
906368d7e2
LF: exhaustive test for CommandPreprocessor (#10914)
While implementing the tests I found a bug.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-17 11:41:29 +00:00
Sofia Faro
61d214e451
Add fetch, exercise implementations for interfaces in speedy. (#10911)
* Draft: Daml Interfaces Speedy PoC

Part of #10810
Extracted from #10670

changelog_begin
changelog_end

* Improve cacheing situation, add implements checks

* scalafmt

* Add comment for ImplementsDefRef

* compile the new update expressions
2021-09-17 10:44:40 +00:00
mziolekda
d01f8e1c35
Fix flaky ApiConfigManagementServiceSpec test (#10922)
* Fix flaky ApiConfigManagementServiceSpec test.

CHANGELOG_BEGIN
CHANGELOG_END

* format code
2021-09-17 08:49:35 +02:00
Robin Krom
50291ed61b
interfaces: scala typechecker implementation (#10867)
* interfaces: scala typechecker implementation

This is the scala side of the lf typechecker for interfaces.

CHANGELOG_BEGIN
CHANGELOG_END

* added collision check

* added exercise/fetch typechecking

* review suggestions

* added todos for collision/typing scala tests
2021-09-16 20:37:33 +00:00
Moritz Kiefer
cac8391d4c
Bump ghc-lib to include daml interfaces parser (#10747)
changelog_begin
changelog_end
2021-09-16 17:52:16 +00:00
tudor-da
b6a6bf7aac
[Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
* Extracted common error implementations
* ErrorGroups
* TransactionError hierarchy
* LedgerApiErrors
* PackageServiceError
* ProtoDeserializationError
* PruningServiceError
* SubmissionErrors

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt extracted error groups and definitions to adhere to local tech stack:
* Use //ledger/error:error core API
* Use DAML SDK logging stack
* Manual rebase to latest changes from Canton

* Extracted RejectionGenerator from Canton

* Adapted RejectionGenerator
* Added //ledger/error:error to artifacts

* Pass correlationId to errors
* Pass parameters as implicits to shave some lines and improve readability

* Workaround for encountered Scala/JDK8 bug

* Addressed Ratko's review comments
2021-09-16 17:19:03 +02:00
Marton Nagy
308f938512
Dpp 494 unit testing ha coordinator (#10862)
* Unit testing: HA Coordinator

* Preparation: switch to fix-thread-pool in AkkaBeforeAndAfterAll to have more stable test runtimes

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: switch to timer from akka-pattern  and Scheduler in waiting-futures, for higher precision timing

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: prevent race in PollingChecker (corner-case uncovered via sustained testing)

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation - HaCoordinator refactorings: switch from DataSource to connection factory functions to clean dependencies, fixing some typos

CHANGELOG_BEGIN
CHANGELOG_END

* Add test infrastructure for locking

CHANGELOG_BEGIN
CHANGELOG_END

* Reuse StorageBackendTestsDBLock to test TestDBLockStorageBackend

CHANGELOG_BEGIN
CHANGELOG_END

* Add unit test suite for HaCoordinator

CHANGELOG_BEGIN
CHANGELOG_END

* Fix random generation to respect scala 2.12

CHANGELOG_BEGIN
CHANGELOG_END

* Adds test cases for graceful-shutdown during initialization

CHANGELOG_BEGIN
CHANGELOG_END

* Minor changes based on review

CHANGELOG_BEGIN
CHANGELOG_END

* Some comment rewording based on review

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-16 17:08:45 +02:00
Stefano Baghino
30f74adc99
Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
changelog_begin
changelog_end
2021-09-16 14:33:34 +00:00
Remy
9582e019ec
LF: Refactor PreprocessorSpec test (#10909)
Just moving stuff arround

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-16 13:58:47 +00:00
Moritz Kiefer
9b0fa29aec
Separate exercise & fetch for interfaces from templates (#10908)
* Separate exercise & fetch for interfaces from templates

part of #10810

changelog_begin
changelog_end

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2021-09-16 12:05:11 +00:00
nicu-da
f4adee91ca
Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
* Add conformance test for command deduplication using the CommandService

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-16 11:16:39 +00:00
Gerolf Seitz
8a391189a6
Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
* Rename Completion.deduplication_time to deduplication_duration

CHANGELOG_BEGIN
CHANGELOG_END

* Breaking protobuf change: regenerate `buf` image

Breaking-Proto: true

* Re-compute sha256s of migrations

* Fix expected error message

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-16 10:41:45 +00:00
Moritz Kiefer
8e22bb6b2d
Drop ContractId typeparameter from Value (#10827)
99% of our usecases use Value[ContractId] so this PR just fixes it.

The few other usescases are:

1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.

We don’t have any code which benefits from being polymorphic in the
contract id type.

changelog_begin
changelog_end
2021-09-16 08:46:57 +00:00
Moritz Kiefer
45437059d4
Drop cocreature as a codeowner from runtime things (#10906)
Just cleaning up my notifications a bit

changelog_begin
changelog_end
2021-09-16 09:38:12 +02:00
Moritz Kiefer
e6e81470bb
Release 1.17.0 RC from second to last commit (#10904)
changelog_begin
changelog_end
2021-09-16 09:10:54 +02:00
Moritz Kiefer
f08ac5f65c
Desugar interface implements declarations (#10895)
* Desugar interface implements declarations

This PR adds desugaring for tplImplements. This consists of the
corresponding typeclass instance (which we just ignore in LF for now,
we probably need it once we have pure functions) and a _implements_
top-level value.

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2021-09-15 19:48:59 +00:00
Hubert Slojewski
b5648c0e3d
Make CommandTracker distinguish submissions of the same command using submissionId [KVL-1104] (#10868)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 14:15:10 +00:00
nicu-da
b4328b3dc3
ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication

CHANGELOG_BEGIN

CHANGELOG_END

* Fix broken test and use consistency for tests

* ledger-api-test-tool - Add conformance test for parallel command deduplication

CHANGELOG_BEGIN
CHANGELOG_END

* Add import for 2.12 compat

* Add silencer plugin

* Split parallel command deduplication scenario into it's own test suite

* Add the parallel command deduplication test to append only ledgers

* Run parallel command deduplication tests for append only ledgers

* Apply suggestions from code review

Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>

* Code review renames

* Add compat import

* Run the test concurrently
2021-09-15 12:15:13 +00:00
Marton Nagy
0c32e3baff
Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing

* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 11:12:17 +00:00
Oliver Seeliger
b3e4975795
Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness

CHANGELOG_BEGIN
CHANGELOG_END

* Make schema migration retries and backoff configurable

* Review feedback - use RetryStrategy.constant instead

* Remove unused tailrec import

* Simplifications and runF by Marton

* Rename config options from retry to attempt and default to 30 attempts
2021-09-15 10:51:03 +00:00
nicu-da
e4cce53957
Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result

The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.

CHANGELOG_BEGIN

CHANGELOG_END

* Do not call duplicate command exception twice
2021-09-15 10:45:23 +00:00
Kamil Bozek
a939594025
Sandbox on H2 - performance improvements for the append-only schema [DPP-600] (#10888)
* Added --database-connection-pool-size parameter to the sandbox CLI

* Improved maximum ledger time lookup query

* Improved active contract lookup query

* Improved max event sequential id query

CHANGELOG_BEGIN
- [Sandbox] - Added a CLI parameter for configuring the number of connections in the database connection pool used for serving ledger API requests
CHANGELOG_END
2021-09-15 12:21:19 +02:00
tudor-da
9a1a1015f2
Increase timeout for heavy tests in ParticipantPruningIT (#10894)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 08:52:33 +00:00
Stefano Baghino
9093c6c7b3
Improve wording for the active contracts service description (#10880)
* Improve wording for the active contracts service description

This should eliminate sources of confusion as mentioned here: https://discuss.daml.com/t/offset-is-null-while-getting-active-contracts/3118

changelog_begin
[Docs] Improved description of the purpose and usage of the active contracts service
changelog_end

* Address https://github.com/digital-asset/daml/pull/10880#pullrequestreview-754078207

* Remove sand-which.
2021-09-15 10:27:40 +02:00
Stefano Baghino
c12f546774
Document #10780 (#10781)
changelog_begin
[Docs/JSON API] documented 256B limitation of Oracle query store
changelog_end
2021-09-15 10:26:28 +02:00
azure-pipelines[bot]
5814f6ad9d
update NOTICES file (#10893)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-09-15 07:30:50 +00:00
tudor-da
38227a8ed7
[Ledger API error codes] ErrorCode enrichments [DPP-591] (#10874)
* [Daml error codes API] Further implementations
* Implements ErrorCode.asGrpcError (and test)
* Error code logging now accepts correlation id and an extra context map
* Full error context is included into enriched logging context

CHANGELOG_BEGIN
CHANGELOG_END

* Fixed Scala 2.12 compilation issues
2021-09-15 07:43:32 +02:00
Brian Healey
e7c443a596
enable json index for all fields that are queried with JSON_EXISTS (#10658)
* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against

CHANGELOG_BEGIN
CHANGELOG_END

* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against
recompute sha

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 18:30:49 -04:00
Gary Verhaegen
6c1c02aeea
document complete authorized auth0 setup (#10881)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 19:11:21 +00:00
Hubert Slojewski
e4230dc51a
Do not drop generated submissionIds in GrpcCommandService [KVL-1104] (#10882)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 17:19:26 +00:00
Gary Verhaegen
b4750a495c
trigger reach auth on internal network (#10844)
In many network setups, there will be a more direct route for the
trigger service to contact the auth middleware than going back through
the frontend's public IP address (and possibly thus also through
intermediaries like an nginx reverse proxy etc.). In _some_ network
setups, it may not even be possible for the trigger service to reach the
auth middleware through its externally-visible address.

This PR caters to these cases by allowing the trigger service to use two
separate addresses for the auth middleware, an internal one the trigger
service uses when it needs to talk to the auth middleware, and an
external one used in generating URLs for external clients.

This is backwards-compatible: if the old option is used, we simply use
the same value for both.

CHANGELOG_BEGIN
- The Trigger Service can now accept separate `--auth-internal` and
  `--auth-external` CLI arguments, where `--auth-internal` is the
  address used by the Trigger Service to reach the Auth Middleware
  directly, and `--auth-external` is the address the Trigger Service uses
  in generated URLs sent back to the client. The `--auth` option remains
  and keeps working as before, setting both internal and external
  addresses to the same given value.
CHANGELOG_END
2021-09-14 14:16:42 +00:00
Gary Verhaegen
79080839c1
add Auth0 support to create-daml-app (#10673)
As I was adding support to a sample app for the third time, I decided I
might as well clean it up a bit and add support for everyone.

This is completely optional, activated by (build-time) environment
variables, and should be trivial to remove for users who don't care
about it. In an ideal world we'd have options to `daml new` and this
would be one, but I think adding it out-of-the-box is a better
alternative than letting people figure this out on their own.

CHANGELOG_BEGIN
- The `create-daml-app` template now includes support for a third
  authentication scheme (in addition to the existing "dev mode" and Daml
  Hub support): Auth0.
CHANGELOG_END
2021-09-14 15:56:45 +02:00
Stefano Baghino
b86490c891
Add @adriaanm-da to the release rotation (#10872)
changelog_begin
changelog_end
2021-09-14 14:54:32 +02:00