Commit Graph

7865 Commits

Author SHA1 Message Date
Hubert Slojewski
0ba54a4bf9
Add a missing test case to the CommandTrackerFlowTest (#10939)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 14:25:42 +02:00
pbatko-da
855ecdf1a7
[DPP-572] Add ledger API test case for verifying conformance to --min-tls-version flag. (#10898)
Add CLI flag to select minimum enabled TLS version for participant server

CHANGELOG_BEGIN
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
CHANGELOG_END
2021-09-20 12:17:00 +02:00
Moritz Kiefer
3e13e3d87e
Switch to stable urls in scoop python manifest (#10933)
The file changed in the master branch which broke our builds :(

changelog_begin
changelog_end
2021-09-20 12:06:58 +02:00
Hubert Slojewski
fc2c87d950
ledger-api-client: Generate a submission id if it's empty in the CommandClient [KVL-1104] (#10926)
* Throw an exception if the submission id is empty in `CommandTracker`

CHANGELOG_BEGIN
CHANGELOG_END

* Generate a submission ID in the `CommandClient` if it's empty

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-20 10:00:57 +02:00
mziolekda
04d8f753b2
Clean up sandbox-on-x conformance tests. (#10766)
* Clean up sandbox-on-x conformance tests.

CHANGELOG_BEGIN
CHANGELOG_END

* fix a flaky test in config management
2021-09-20 07:52:46 +00:00
Oliver Seeliger
b4541b5a4d
Logging delay of submitted commands (#10912)
* Logging delay of submitted commands

This would have helped track down the timezone issue more quickly.

CHANGELOG_BEGIN
CHANGELOG_END

* Review feedback
2021-09-20 06:43:22 +00:00
nicu-da
b8e21d8f48
Fix takeFilter for the test StreamConsumer (#10918)
* Reorder the observers for StreamConsumer to first filter and then take just one element, not the other way around

CHANGELOG_BEGIN
CHANGELOG_END

* Refactor the test to get better stacktraces during failures

* Remove unused import
2021-09-17 16:16:04 +02:00
Remy
906368d7e2
LF: exhaustive test for CommandPreprocessor (#10914)
While implementing the tests I found a bug.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-17 11:41:29 +00:00
Sofia Faro
61d214e451
Add fetch, exercise implementations for interfaces in speedy. (#10911)
* Draft: Daml Interfaces Speedy PoC

Part of #10810
Extracted from #10670

changelog_begin
changelog_end

* Improve cacheing situation, add implements checks

* scalafmt

* Add comment for ImplementsDefRef

* compile the new update expressions
2021-09-17 10:44:40 +00:00
mziolekda
d01f8e1c35
Fix flaky ApiConfigManagementServiceSpec test (#10922)
* Fix flaky ApiConfigManagementServiceSpec test.

CHANGELOG_BEGIN
CHANGELOG_END

* format code
2021-09-17 08:49:35 +02:00
Robin Krom
50291ed61b
interfaces: scala typechecker implementation (#10867)
* interfaces: scala typechecker implementation

This is the scala side of the lf typechecker for interfaces.

CHANGELOG_BEGIN
CHANGELOG_END

* added collision check

* added exercise/fetch typechecking

* review suggestions

* added todos for collision/typing scala tests
2021-09-16 20:37:33 +00:00
Moritz Kiefer
cac8391d4c
Bump ghc-lib to include daml interfaces parser (#10747)
changelog_begin
changelog_end
2021-09-16 17:52:16 +00:00
tudor-da
b6a6bf7aac
[Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
* Extracted common error implementations
* ErrorGroups
* TransactionError hierarchy
* LedgerApiErrors
* PackageServiceError
* ProtoDeserializationError
* PruningServiceError
* SubmissionErrors

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt extracted error groups and definitions to adhere to local tech stack:
* Use //ledger/error:error core API
* Use DAML SDK logging stack
* Manual rebase to latest changes from Canton

* Extracted RejectionGenerator from Canton

* Adapted RejectionGenerator
* Added //ledger/error:error to artifacts

* Pass correlationId to errors
* Pass parameters as implicits to shave some lines and improve readability

* Workaround for encountered Scala/JDK8 bug

* Addressed Ratko's review comments
2021-09-16 17:19:03 +02:00
Marton Nagy
308f938512
Dpp 494 unit testing ha coordinator (#10862)
* Unit testing: HA Coordinator

* Preparation: switch to fix-thread-pool in AkkaBeforeAndAfterAll to have more stable test runtimes

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: switch to timer from akka-pattern  and Scheduler in waiting-futures, for higher precision timing

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: prevent race in PollingChecker (corner-case uncovered via sustained testing)

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation - HaCoordinator refactorings: switch from DataSource to connection factory functions to clean dependencies, fixing some typos

CHANGELOG_BEGIN
CHANGELOG_END

* Add test infrastructure for locking

CHANGELOG_BEGIN
CHANGELOG_END

* Reuse StorageBackendTestsDBLock to test TestDBLockStorageBackend

CHANGELOG_BEGIN
CHANGELOG_END

* Add unit test suite for HaCoordinator

CHANGELOG_BEGIN
CHANGELOG_END

* Fix random generation to respect scala 2.12

CHANGELOG_BEGIN
CHANGELOG_END

* Adds test cases for graceful-shutdown during initialization

CHANGELOG_BEGIN
CHANGELOG_END

* Minor changes based on review

CHANGELOG_BEGIN
CHANGELOG_END

* Some comment rewording based on review

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-16 17:08:45 +02:00
Stefano Baghino
30f74adc99
Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
changelog_begin
changelog_end
2021-09-16 14:33:34 +00:00
Remy
9582e019ec
LF: Refactor PreprocessorSpec test (#10909)
Just moving stuff arround

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-16 13:58:47 +00:00
Moritz Kiefer
9b0fa29aec
Separate exercise & fetch for interfaces from templates (#10908)
* Separate exercise & fetch for interfaces from templates

part of #10810

changelog_begin
changelog_end

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2021-09-16 12:05:11 +00:00
nicu-da
f4adee91ca
Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
* Add conformance test for command deduplication using the CommandService

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-16 11:16:39 +00:00
Gerolf Seitz
8a391189a6
Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
* Rename Completion.deduplication_time to deduplication_duration

CHANGELOG_BEGIN
CHANGELOG_END

* Breaking protobuf change: regenerate `buf` image

Breaking-Proto: true

* Re-compute sha256s of migrations

* Fix expected error message

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-16 10:41:45 +00:00
Moritz Kiefer
8e22bb6b2d
Drop ContractId typeparameter from Value (#10827)
99% of our usecases use Value[ContractId] so this PR just fixes it.

The few other usescases are:

1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.

We don’t have any code which benefits from being polymorphic in the
contract id type.

changelog_begin
changelog_end
2021-09-16 08:46:57 +00:00
Moritz Kiefer
45437059d4
Drop cocreature as a codeowner from runtime things (#10906)
Just cleaning up my notifications a bit

changelog_begin
changelog_end
2021-09-16 09:38:12 +02:00
Moritz Kiefer
e6e81470bb
Release 1.17.0 RC from second to last commit (#10904)
changelog_begin
changelog_end
2021-09-16 09:10:54 +02:00
Moritz Kiefer
f08ac5f65c
Desugar interface implements declarations (#10895)
* Desugar interface implements declarations

This PR adds desugaring for tplImplements. This consists of the
corresponding typeclass instance (which we just ignore in LF for now,
we probably need it once we have pure functions) and a _implements_
top-level value.

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2021-09-15 19:48:59 +00:00
Hubert Slojewski
b5648c0e3d
Make CommandTracker distinguish submissions of the same command using submissionId [KVL-1104] (#10868)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 14:15:10 +00:00
nicu-da
b4328b3dc3
ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication

CHANGELOG_BEGIN

CHANGELOG_END

* Fix broken test and use consistency for tests

* ledger-api-test-tool - Add conformance test for parallel command deduplication

CHANGELOG_BEGIN
CHANGELOG_END

* Add import for 2.12 compat

* Add silencer plugin

* Split parallel command deduplication scenario into it's own test suite

* Add the parallel command deduplication test to append only ledgers

* Run parallel command deduplication tests for append only ledgers

* Apply suggestions from code review

Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>

* Code review renames

* Add compat import

* Run the test concurrently
2021-09-15 12:15:13 +00:00
Marton Nagy
0c32e3baff
Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing

* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 11:12:17 +00:00
Oliver Seeliger
b3e4975795
Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness

CHANGELOG_BEGIN
CHANGELOG_END

* Make schema migration retries and backoff configurable

* Review feedback - use RetryStrategy.constant instead

* Remove unused tailrec import

* Simplifications and runF by Marton

* Rename config options from retry to attempt and default to 30 attempts
2021-09-15 10:51:03 +00:00
nicu-da
e4cce53957
Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result

The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.

CHANGELOG_BEGIN

CHANGELOG_END

* Do not call duplicate command exception twice
2021-09-15 10:45:23 +00:00
Kamil Bozek
a939594025
Sandbox on H2 - performance improvements for the append-only schema [DPP-600] (#10888)
* Added --database-connection-pool-size parameter to the sandbox CLI

* Improved maximum ledger time lookup query

* Improved active contract lookup query

* Improved max event sequential id query

CHANGELOG_BEGIN
- [Sandbox] - Added a CLI parameter for configuring the number of connections in the database connection pool used for serving ledger API requests
CHANGELOG_END
2021-09-15 12:21:19 +02:00
tudor-da
9a1a1015f2
Increase timeout for heavy tests in ParticipantPruningIT (#10894)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 08:52:33 +00:00
Stefano Baghino
9093c6c7b3
Improve wording for the active contracts service description (#10880)
* Improve wording for the active contracts service description

This should eliminate sources of confusion as mentioned here: https://discuss.daml.com/t/offset-is-null-while-getting-active-contracts/3118

changelog_begin
[Docs] Improved description of the purpose and usage of the active contracts service
changelog_end

* Address https://github.com/digital-asset/daml/pull/10880#pullrequestreview-754078207

* Remove sand-which.
2021-09-15 10:27:40 +02:00
Stefano Baghino
c12f546774
Document #10780 (#10781)
changelog_begin
[Docs/JSON API] documented 256B limitation of Oracle query store
changelog_end
2021-09-15 10:26:28 +02:00
azure-pipelines[bot]
5814f6ad9d
update NOTICES file (#10893)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-09-15 07:30:50 +00:00
tudor-da
38227a8ed7
[Ledger API error codes] ErrorCode enrichments [DPP-591] (#10874)
* [Daml error codes API] Further implementations
* Implements ErrorCode.asGrpcError (and test)
* Error code logging now accepts correlation id and an extra context map
* Full error context is included into enriched logging context

CHANGELOG_BEGIN
CHANGELOG_END

* Fixed Scala 2.12 compilation issues
2021-09-15 07:43:32 +02:00
Brian Healey
e7c443a596
enable json index for all fields that are queried with JSON_EXISTS (#10658)
* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against

CHANGELOG_BEGIN
CHANGELOG_END

* fixes #9975 - enable json index for all fields that we run JSON_EXISTS queries against
recompute sha

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 18:30:49 -04:00
Gary Verhaegen
6c1c02aeea
document complete authorized auth0 setup (#10881)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 19:11:21 +00:00
Hubert Slojewski
e4230dc51a
Do not drop generated submissionIds in GrpcCommandService [KVL-1104] (#10882)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 17:19:26 +00:00
Gary Verhaegen
b4750a495c
trigger reach auth on internal network (#10844)
In many network setups, there will be a more direct route for the
trigger service to contact the auth middleware than going back through
the frontend's public IP address (and possibly thus also through
intermediaries like an nginx reverse proxy etc.). In _some_ network
setups, it may not even be possible for the trigger service to reach the
auth middleware through its externally-visible address.

This PR caters to these cases by allowing the trigger service to use two
separate addresses for the auth middleware, an internal one the trigger
service uses when it needs to talk to the auth middleware, and an
external one used in generating URLs for external clients.

This is backwards-compatible: if the old option is used, we simply use
the same value for both.

CHANGELOG_BEGIN
- The Trigger Service can now accept separate `--auth-internal` and
  `--auth-external` CLI arguments, where `--auth-internal` is the
  address used by the Trigger Service to reach the Auth Middleware
  directly, and `--auth-external` is the address the Trigger Service uses
  in generated URLs sent back to the client. The `--auth` option remains
  and keeps working as before, setting both internal and external
  addresses to the same given value.
CHANGELOG_END
2021-09-14 14:16:42 +00:00
Gary Verhaegen
79080839c1
add Auth0 support to create-daml-app (#10673)
As I was adding support to a sample app for the third time, I decided I
might as well clean it up a bit and add support for everyone.

This is completely optional, activated by (build-time) environment
variables, and should be trivial to remove for users who don't care
about it. In an ideal world we'd have options to `daml new` and this
would be one, but I think adding it out-of-the-box is a better
alternative than letting people figure this out on their own.

CHANGELOG_BEGIN
- The `create-daml-app` template now includes support for a third
  authentication scheme (in addition to the existing "dev mode" and Daml
  Hub support): Auth0.
CHANGELOG_END
2021-09-14 15:56:45 +02:00
Stefano Baghino
b86490c891
Add @adriaanm-da to the release rotation (#10872)
changelog_begin
changelog_end
2021-09-14 14:54:32 +02:00
Moisés Ackerman
9e918c37a3
Update trigger-service docs to use --dar option in the corresponding example (#10877)
changelog_begin
changelog_end
2021-09-14 14:46:31 +02:00
Artúr Poór
49a9556e5c
[docs] Fix minor typo in doc of exerciseByKey in TS. (#10863)
changelog_begin
changelog_end
2021-09-14 12:30:50 +00:00
Robin Krom
f7c07eaa0c
interfaces: scala protobuf encoder (#10878)
This implements the protobuf encoder of the scala side for interfaces.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 12:27:53 +00:00
fabiotudone-da
be4e06427f
Ledger API Test Tool: support --additional tests [KVL-1100] (#10829)
* Support adding tests as an hidden option

* Simplify existing suites

CHANGELOG_BEGIN
CHANGELOG_END

* Remove stale conformance suites from build.yml

* `--add` -> ``--additional`

* Re-add `--all-tests` as deprecated CLI option to be tested

* Move sandbox-classic pruning test to wall clock again

* Run KVCommandDeduplicationIT for sandbox append-only

* Tidy-up

* Also add participant pruning test to ledger-on-memory/single-participant

* Remove KVCommandDeduplicationIT on ledger-on-memory/append-only

* Run the full suite plus pruning (rather than just pruning) for ledger-on-memory with multiple participants and append-only

* Add KVCommandDeduplicationIT to ledger-on-memory append-only

* Exclude ConfigManagementServiceIT from ledger-on-memory append-only multi-participant

* Tidy-up

* Use KVCommandDeduplicationIT for sandbox-on-x too

* Fix merge

Add max dedup duration arg to all the test suites that include command dedup tests

* Make `--include` and `--additional` mutually exclusive

* Uniform formatting of multi-line strings

* Move exclusions after additions as they are applied last

* Re-disable deduplication test on sandbox with static time

* Re-disable deduplication test on sandbox-on-x
2021-09-14 14:00:30 +02:00
tudor-da
97e14de644
[Ledger API error codes] ErrorCode interfaces and generator [DPP-591] (#10836)
* [Ledger API error codes] ErrorCode interfaces and generator
* Implementation of annotation processor
* Implementation of DocItem generator
* Unit testing of the generator and error code logging

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments
2021-09-14 13:50:00 +02:00
pbatko-da
6dcdaa411c
[DPP-589] Add CLI flag to select minimum enabled TLS version (#10854)
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox

CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
2021-09-14 12:37:38 +02:00
matevarga-da
1fc58d93f1
Navigator customviews highlight and choices button, apply custom theme on the login screen (#10859)
* Custom views menu highlighting and choices button fixes.

* Consider theme on the login screen

* Consider theme on the login screen

Originally reported here: https://discuss.daml.com/t/how-to-highlight-selected-custom-filter-button-in-navigator-how-to-use-the-choice-field-in-a-custom-view

changelog_begin
- [Navigator] The currently selected custom view is now highlighted on the sidebar
changelog_end

* Use page.state.id and defined user object.
2021-09-14 11:32:04 +02:00
Miklos
6faddc90c8
Update Daml Documentation to reflect command deduplication related changes [KVL-1094] (#10852)
* Minor tweaks.

* Use deduplication period instead of deduplication time.

* Introduced change ID and consistent use of deduplication duration/period.

* Consistent use of deduplication duration/period.

* Added ALREADY_EXISTS to the described gRPC errors.

* Minor tweak.

CHANGELOG_BEGIN
CHANGELOG_END

* Emphasize that applications should not change the deduplication time upon a resubmission

* Fix definition of change ID and always refer to it

* Try to clarify "Application-specific IDs"

* Make command deduplication explanation unspecific w.r.t. direction

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-14 11:28:27 +02:00
nickchapman-da
7c29eee130
Cleanup normalize from svalue (#10873)
* better variable name

CHANGELOG_BEGIN
CHANGELOG_END

* remove disallowGenMapAtVersion check
2021-09-14 09:23:48 +00:00
nickchapman-da
053f22a1af
Convert SValue to Value, and normalize, in a single code pass. (#10828)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-14 08:07:26 +01:00