Commit Graph

2185 Commits

Author SHA1 Message Date
mziolekda
60e372dcd2
Don't run pruning tests on H2, they time-out (#11897)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 13:06:03 +00:00
Remy
58e69ade1a
LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
the more consistent, as asked by Moritz in review of #11820

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 12:23:17 +00:00
nicu-da
a1705d669a
participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838)
changelog_begin
changelog_end
2021-11-26 11:12:17 +00:00
pbatko-da
c27406cdcf
[DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879
Other changes:
- Fix ErrorCodeDocumentationGeneratorSpec not being in any bazel target.
- (partially) Fix inconsistent usage of deprecated, Deprecation and DeprecatedDocs annotation.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 10:22:14 +01:00
tudor-da
abc141b57e
Increase pruning tests timeout (#11891)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 08:16:35 +00:00
mziolekda
056fc52706
Log while processing base64 encoded server key [DPP-761] (#11835)
* Log while processing base64 encoded server key

CHANGELOG_BEGIN
CHANGELOG_END

* address review comments

* demote logs from error to debug

* format
2021-11-25 18:02:35 +00:00
Kamil Bozek
f69bd688cc
ledger-api-bench-tool: Fix flaky MetricsCollectorSpec (#11750)
* Renamed MetricReporter to MetricFormatter

* Make MetricsCollector communicate via messages only, not by logging

* Logging reports from MetricManger

* Cancel periodic report request when stream is done

* Removed debug sleep

* Fix error when defining reporting period less than one second

* Fixed flaky MetricsCollectorSpec test

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 17:39:30 +01:00
Victor Peter Rouven Müller
df373466dc
[JSON-API] Add query store metrics (#11809)
* Add query store metrics

CHANGELOG_BEGIN

- [JSON-API] added metrics to separately track:
    - time taken to update query-store ACS (from ledger)
    - lookup times for the query store

CHANGELOG_END

* Apply review comment
2021-11-25 13:04:28 +00:00
pbatko-da
5f3a4d2067
[Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867)
Also make section header markers consistent docs' reamde.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 13:46:12 +01:00
tudor-da
cf3ac011ca
[Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828)
* [Self-service error codes] Do not return error code id in metadata

CHANGELOG_BEGIN
CHANGELOG_END

* Do not propagate definite_answer
* Additionally, remove redundant context info from error codes

* Fix tests

* Create IndexDbException as a specialization
* Used to globally define error codes returned by the persistence layer
that are logging
2021-11-25 12:42:29 +01:00
Marton Nagy
beca0ee3e9
Refactor StandaloneApiServer factory (#11842)
To enable direct IndexServer building (and enable two step ApiServer build process)
This will be needed for KV WriteService proxy: a wrapper is being build to serve deduplication time conversion, which needs access to completion service.

changelog_begin
changelog_end
2021-11-25 10:12:23 +01:00
Sofia Faro
5c12d757f4
Add a guard when exercising by interface. (#11836)
* Add a guard when exercising by interface.

This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.

It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.

I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)

changelog_begin
changelog_end

* scalafmt

* Feedback and fix matches

* Update comments, we are always going to abort the transaction

* Raise WronglyTypedContract in SBGuardTemplateId.

* rebase and fix parser

* restore ANF

* scalafmt
2021-11-24 14:22:40 +00:00
fabiotudone-da
7c3a2a7801
Add a new KV submission failure error (#11854)
CHANGELOG_BEGIN
[Integration Kit] Add a new SUBMISSION_FAILED internal error
CHANGELOG_END
2021-11-24 13:10:36 +00:00
Robert Autenrieth
aebc5a734f
All packages must be valid (#11850)
changelog_begin
changelog_end
2021-11-24 13:10:27 +00:00
Robert Autenrieth
25b476fdb2
DPP-726 Add string interning unit tests (#11841)
* DPP-726 Add string interning unit tests

changelog_begin
changelog_end

* Add more test cases
2021-11-24 13:38:38 +01:00
nicu-da
59eb0d2eff
kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848)
changelog_begin
kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`.
changelog_end
2021-11-24 04:32:34 -08:00
Hubert Slojewski
109b60613c
Make the InstrumentedSource.queue use the BoundedSourceQueue [KVL-1177] (#11807)
CHANGELOG_BEGIN

- [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future

CHANGELOG_END
2021-11-24 11:26:39 +01:00
Robert Autenrieth
bd2a6852ee
Make ACS reader parameters configurable (#11732)
changelog_begin
changelog_end
2021-11-23 21:48:02 +01:00
tudor-da
329e609ec4
[Self-service error codes] Group explanations (#11715)
* [Self-service error codes] Error group descriptions

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-23 20:30:33 +00:00
pbatko-da
9461702309
[DPP-673][Self-service error codes] Split one narrow migration table into multiple narrower ones to fit in the generated pdf's pages. (#11803)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 17:38:31 +01:00
pbatko-da
55191847cd
[DPP-673][Self-service error-codes] Generate error code directory compatible with both pdf and html docs (#11794)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 15:30:48 +01:00
mziolekda
f2aa09c114
Allow encoded server key files to be in base64 [DPP-761] (#11796)
* Allow encoded server key files to be in base64

CHANGELOG_BEGIN
CHANGELOG_END

* format

* dry tests
2021-11-23 09:49:52 +00:00
tudor-da
1bfcbc978b
Refactoring for Canton compatibility on PackageServiceErrors (#11812)
* Refactoring for Canton compatibility on PackageServiceErrors
* ErrorResource.all is public for accessibility from Canton

CHANGELOG_BEGIN
CHANGELOG_END

* Keep PackageServiceError object
2021-11-22 19:14:03 +00:00
Moritz Kiefer
addb2edc03
Drop Scala 2.13 version guards from Bazel definitions (#11819)
changelog_begin
changelog_end
2021-11-22 17:37:49 +00:00
tudor-da
8e31e8e6a8
[Self-service error codes] Enabled by default [DPP-691] (#11685)
* [Self-service error codes] Enabled by default
* Flag changed to `use-pre-1.18-error-codes` (disabled by default)

CHANGELOG_BEGIN
[Ledger API Specification] The Ledger API returns enriched error codes (see https://docs.daml.com/error-codes/self-service/index.html)
For backwards-compatibility, a new API flag `--use-pre-1.18-error-codes` is introduced for preserving the legacy behavior for
clients that want to migrate incrementally to the changed gRPC status code responses and error details format.
CHANGELOG_END

* Adapted HttpServiceIntegrationTest

* Renamed `Feature Flag` to `Configuration` in docs

* Fix Daml Script tests

changelog_begin
changelog_end

* Fix Repl functests

changelog_begin
changelog_end

* Fix haskell binding tests

changelog_begin
changelog_end

* Fix CommandClientIT test

* Fixed Sandbox and CommandServiceBackpressureIT tests
 Please enter the commit message for your changes. Lines starting

* Adapt //compiler/damlc/tests:repl-functests again

* Fix more tests and address Miklos' comments
* Flag name changed to `grpc-status-codes-compatibility-mode`

* Remove useless flags sandbox-classic

* Sandbox-classic tests fix for ContractKeysIT and ExceptionsIT
* Created 2 deprecated test suites that have the more generic assertions as returned
by the deprecated in-memory backend

* More fixes for CommandServiceIT
* Fixes compilation issue with the deprecated exceptionsIT class for Sandbox-classic in-memory

* Compatibility mode for old test tools

* Change flag name to `use-pre-1.18-error-codes`

* Apply suggestions from code review

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-11-22 12:26:48 +00:00
nicu-da
2d9ee2078f
[ledger-api-client] - Do not account for deduplication_time as timeout [KVL-1172] (#11791)
changelog_begin
ledger-api-client - The default command tracking timeout is no longer influenced by the deprecated deduplication_time as a deduplication period. Previously if no timeout was being set in the command and deduplication_time was set as the deduplication period then the command tracking timeout was the minimum between the deduplication_time and max tracking timeout.
changelog_end
2021-11-19 18:23:27 +00:00
nicu-da
25e5bbb753
kvutils - Removed unused setting from conformance tests [KVL-1174] (#11785)
kvutils - The deduplication period is no longer overwritten by the max deduplication duration, therefore there is no need to change the default max deduplication duration

changelog_begin
changelog_end
2021-11-19 05:41:45 -08:00
Marton Nagy
4b59c5731c
Move select DB logic to JVM [DPP-760] (#11777)
* Move select logic which was previously added as select expressions to Scala calulation, and exposing needed data on the queries

changelog_begin
changelog_end
2021-11-18 21:36:15 +01:00
mziolekda
a9754fe7d2
Write exception reports into the logfile in case of ssl misconfiguration (#11776)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-18 18:25:59 +00:00
pbatko-da
2e789ddf37
[DPP-417][Self-service error codes][Docs] Add more complete information for forking gRPC status codes. (#11739)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-18 17:03:10 +01:00
nicu-da
4ebdaf597d
kvutils - Deprecate deduplicate_until [kvl-1174] (#11765)
[kvutils] - deduplicate_until is no longer used by backwards looking command deduplication, therefore we deprecate the field in the proto files (keep it there for backwards compatibility) and remove references to it from the committer context

changelog_begin
changelog_end
2021-11-18 07:45:54 -08:00
nicu-da
f5a6302ba1
[participant-integration-api] - Use internal CommandCompletionService [KVL-1172] (#11741)
Fully decouple the usage of the gRPC CommandCompletionService vs the internally defined CommandCompletionService. This allows the two interfaces to diverge and does not force us to expose all the methods that we want to add to the internal CommandCompletionService.

changelog_begin
changelog_end
2021-11-18 07:19:27 -08:00
Andreas Herrmann
39a38d3a37
Update to Java 11 (#11512)
* Update to Java 11

changelog_begin
changelog_end

* Fix RoundingMode deprecation warnings

* Fix dep-ann warning

* Integer constructor

* JavaX annotation dependency

* javax.xml.bind was removed in Java 11

Using Guava as a replacement, since it is already a project dependency.

* JDK 11 no longer has a separate JRE tree

* Remove unused jdk_nix import

* remove now redundant jdk11_nix

* Java 8 --> 9 increased Instant.now() precision

See https://bugs.openjdk.java.net/browse/JDK-8068730

The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.

`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.

* TMP round timestamps

* Revert "TMP round timestamps"

This reverts commit af8e261278.

* Skip versions before 1.6.0 in migration tests

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-18 14:48:37 +00:00
nicu-da
8f458d8d2a
[ledger-api] Include the completion offset in the command_service.proto responses [KVL-1171] (#11658)
CHANGELOG_BEGIN
[ledger-api] - Include the completion offset in the responses from the command_service.proto
CHANGELOG_END
2021-11-18 04:37:26 -08:00
Samir Talwar
e891180fb6
participant-state: Remove the v1 API. (#11553)
CHANGELOG_BEGIN
- [Integration Kit] v1 of the participant state API is no longer
  supported. You will need to upgrade to v2.
CHANGELOG_END
2021-11-18 11:49:33 +00:00
nicu-da
dd88ba24c3
kvutils - Backwards looking command deduplication [KVL-1174] (#11634)
CHANGELOG_BEGIN
kvutils - The deduplication duration passed in the command is now used for command deduplication, and is no longer always overwritten with the max deduplication duration. The command deduplication duration can still be extended by the committer to account for time skews.
CHANGELOG_END
2021-11-18 02:25:31 -08:00
Samir Talwar
43438c6e24
sandbox: Remove PostgreSQL conformance tests. (#11434)
PostgreSQL is no longer supported. Let's not worry about it too much.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-18 10:06:18 +00:00
tudor-da
b0a1bf79df
More missing error codes (#11749)
* Missing error codes in Sandbox classic

CHANGELOG_BEGIN
CHANGELOG_END

* Covering more errors above the persistence layer

* Fix wrong contextualizedErrorLoggers

* Compilation fixes
2021-11-18 10:00:43 +00:00
Moritz Kiefer
3366413dd4
Delete empty scala test library (#11747)
changelog_begin
changelog_end
2021-11-18 10:22:35 +01:00
Moritz Kiefer
c8ace8b1cf
Factor kind projector target into a variable (#11759)
Seems nicer than trying to format a string everywhere

changelog_begin
changelog_end
2021-11-18 09:51:43 +01:00
Moritz Kiefer
fa7663148a
Drop 2.12 versioned_scala_deps (#11748)
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.

changelog_begin
changelog_end
2021-11-17 22:13:08 +00:00
tudor-da
a05a40aeec
[Self-service error codes] Adapt ResetService in Sandbox classic (#11731)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-17 11:11:54 +01:00
Marton Nagy
eca53e9ede
Integrate ACSReader [DPP-720] (#11688)
* Integrating ACSReader and 2 phase retrieval logic to TransactionReader
* Add metrics and debug logging

changelog_begin
changelog_end
2021-11-17 09:46:24 +00:00
tudor-da
fbc436d70a
[DPP-739][Self-service error codes] Adapt QueueBackedTracker error codes (#11719)
* [Self-service error codes] Adapt QueueBasedTracker

CHANGELOG_BEGIN
CHANGELOG_END

* Added changed error codes to migration guide

* Fixed ErrorFactoriesSpec

* Update ledger/error/src/main/scala/com/daml/error/definitions/LedgerApiErrors.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: Pawel Batko <pawel.batko@digitalasset.com>
Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-16 22:21:29 +00:00
pbatko-da
07cff7cdb7
[DPP-417][Self-service error-codes] Document error categories. (#11727)
* [Self-service error-codes] Docs intro WIP

* CHANGELOG_BEGIN
CHANGELOG_END

* 1

* Fix rst

CHANGELOG_BEGIN
CHANGELOG_END

* [DPP-417][Self-service error-codes] Document error categories.

CHANGELOG_BEGIN
CHANGELOG_END

* 1

* Fix
2021-11-16 22:55:35 +01:00
Marton Nagy
c866fa6701
Optimize data migrations for performance [DPP-714] (#11714)
* Refactoring interning data migrations for performance
* Planned improvement: migration-verification

changelog_begin
changelog_end
2021-11-16 19:43:22 +00:00
Miklos
a63e091068
kvutils: Use consistent argument name casing in KVErrors (#11609)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 17:30:29 +00:00
tudor-da
c1f4beb328
Ledger ID mismatch does not change error codes. (#11720)
* Remains NOT_FOUND

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 17:05:09 +00:00
pbatko-da
a29fee925c
[Self-service error-codes] Add introductory section to self-service error codes page. (#11712) 2021-11-16 16:41:41 +01:00
tudor-da
05f604d672
[Self-service error codes] Enrich migration guide [DPP-755] (#11711)
* Adapted migration guide to include changes from https://github.com/digital-asset/daml/pull/11686

CHANGELOG_BEGIN
CHANGELOG_END

* Enriched migration guide with Sandbox-classic changed gRPC error codes

* Added migration guide for error codes changed in KV committer

* Extract the KV and Sandbox-classic migration guides

* Hierarchical order of error codes docs

* Added back full grpc error code description to columns
* Using "gRPC status code" instead of "gRPC error code"
* Addressed review comments from Hubert

* Apply suggestions from code review

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* Addressed review comments
* Use 'Error codes' instead of 'Self-service error codes'

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-11-16 14:25:53 +00:00