Commit Graph

859 Commits

Author SHA1 Message Date
Marton Nagy
f5a2539d6f
Add globalKeyMapping to WriteService [DPP-1067] (#14026)
The field globalKeyMapping is recently added to
lf.transaction.Transaction.Metadata, and which is
computed by the Engine.
This field might be needed for processing by WriteService
implementors.
This PR adds this to the WriteService.

changelog_begin
Global contract key mapping is added to WriteService.
changelog_end
2022-06-01 10:08:18 +02:00
Nicu Reut
a792de39a5
Update scalafmt to 3.5.1. (#13584)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-28 09:57:04 +00:00
Hubert Slojewski
61be3a5e39
Move kvutils out of the SDK [KVL-1245] (#13704) 2022-04-27 15:21:28 +02:00
Hubert Slojewski
d700fc9e76
Move kvutils test dar file lib to participant-state [KVL-1245] (#13696)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 13:42:04 +02:00
Hubert Slojewski
0e9326f857
Publish daml-lf/snapshot proto [KVL-1245] (#13679)
* Publish daml-lf/snapshot proto [KVL-1245]

CHANGELOG_BEGIN
CHANGELOG_END

* fix snapshot_proto bazel target

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2022-04-26 07:59:12 +02:00
Remy
3be994c4e8
LF: introduce interfaceId field in Exercise node (#13643)
.. to track where the choice is defined.

part of #13653

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-21 13:19:25 +00:00
Nicu Reut
a37dd63675
Remove committer support for non pre-execution [kvl-734] (#13598)
CHANGELOG_BEGIN
[kvutils] - Pre-execution is the only supported execution mode
CHANGELOG_END
2022-04-20 09:52:53 +00:00
Hubert Slojewski
16a97e8715
Move KVErrors to participant-state-kv-errors [KVL-1371] (#13604)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-19 15:22:31 +02:00
Sergey Kisel
a6b3d5f581
Cleanup around configuration (#13597)
* Cleanup around configuration

CHANGELOG_BEGIN
CHANGELOG_END

* Removal of unused configuration parameters. Merge cohesive params.

* Removal of ParticipantIndexerConfig as redundant transformation layer.

* Extracting engine config values

* Separation of IndexConfiguration
2022-04-19 13:19:11 +02:00
Nicu Reut
988b609051
Ensure out of time bounds entries are set for all rejections [KVL-1412] (#13595) 2022-04-18 12:04:49 +02:00
Nicu Reut
2ad6d2985f
[kvutils] Remove unused batched submission validation [KVL-1372][KVL-734] (#13589) 2022-04-14 15:56:26 +02:00
Nicu Reut
c8f4a84eed
[kvutils] Remove the kv integrity checker [kvl-1365] (#13585) 2022-04-14 10:39:29 +02:00
Remy
4e51194041
LF: Drop byInterface field from Transaction Nodes (#13520)
part of #13491

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-07 11:06:17 +02:00
Hubert Slojewski
a8547c8bab
Make multi-participant conformance tests work with distributed committers/event logs [KVL-1335] (#13467)
CHANGELOG_BEGIN
CHANGELOG_END
2022-03-31 14:24:06 +02:00
Remy
f7d1901a5c
LF: split Command into ApiCommand and ReplayCommand (#13323)
CHANGELOG_BEGIN
CHANGELOG_END
2022-03-21 14:07:41 +01:00
Remy
6c2dc2d508
Move Replay from KV to LF land (#13243)
* Move Replay from KV to LF land

+ base it on submission entries instead of ledger export

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-21 13:03:10 +01:00
pbatko-da
35c5812703
[Error codes] Update docstring for com.daml.ledger.participant.state.v2.PruningResult.NotPruned [DPP-861] (#13305)
changelog_begin
changelog_end
2022-03-16 15:07:42 +01:00
pbatko-da
c6a0275f6e
[Error codes] Fill in missing contexts that were lost where reflection based context population was removed [DPP-606] (#13264)
And rename error arguments that haven't been in camel case yet.

changelog_begin
changelog_end
2022-03-16 15:07:24 +01:00
Sergey Kisel
a4f4300502
Naming and logging improvements [DPP-956] (#13270)
* Naming and logging improvements [DPP-956]

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-15 09:40:39 +01:00
pbatko-da
b663a1c1ec
WriteParticipantPruningService propagates rich error codes on prune failure [DPP-861] (#13238)
changelog_begin
changelog_end
2022-03-10 14:48:28 +01:00
pbatko-da
fbb3ba55c2
Simplify implementations of BaseError trait [DPP-606] (#13185)
1. Remove `BaseError.Impl`, `LoggingTransactionErrorImpl` and `LoggingPackageServiceError` 
and instead provide more direct `DamlError` and `DamlErrorWithDefiniteAnswer`.


2. Remove custom implementation of `TransactionError.rpcStatus` and instead provide simpler one in `DamlErr.r.rpcStatus` (which works by first calling `code.asGrpcStatus` and then converting the result to `com.google.rpc.status.Status`).

3. Remove `GrpcStatus.toProto` and instead use `DamlError.rpcStatus`.

4. Use `asGrpcStatus` and `asGrpcError`  instead of `asGrpcStatusFromContext` and `asGrpcErrorFromContext` where possible.


changelog_begin
changelog_end
2022-03-10 12:12:14 +01:00
Remy
8abf0bae96
Generate SubmissionEntry from KV export (#13209)
We split kv replay as follow:

- [LF] we introduce a simple protobuf to store submission entry (which
  roughly correspond to TRANSACTION_ENTRY and PACKAGE_UPLOAD_ENTRY KV
  submission)

- [KV] we create a tool to extract form a ledger export a sequence of
  submission entry

- [LF] we move the replay tool from KV to LF, and base it on
  submission entries instead of ledger export

The objectify of this split is double:

- on the one side it will simplify maintenance, as it will make API
  between KV and LF more clear (depends only on a protobuf definition)

- on the other side, it will made the use of the tool handy, as it
  will make possible to benchmark the engine without direct dependency
  between KV and LF.

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-10 08:47:04 +01:00
Sergey Kisel
9a27edd656
Improve MessageDigest and Mac instance creation to solve lock contention [DPP-956] (#13221)
* Improve MessageDigest and Mac instance creation to solve lock contention problem [DPP-956]

changelog_begin
Scalability bottleneck in regard to hashing has been fixed in multiple places.
changelog_end
2022-03-09 22:15:32 +01:00
Simon Maxen
cad7928220
Metering cleanup [DPP-937] (#13210)
* Make no-op metering logging debug

* Address statistic defaulting TODOs

* TODO replaced with ticket

* Switch to using assertLedgerMeteringEnd when ledgering end must be populated

* Note in docs that ledger metering is not affected by participant pruning

* Address review comments leftover from DPP-819

* Switch to using cSQL for options and offsets

changelog_begin
changelog_end
2022-03-09 19:06:04 +00:00
pbatko-da
c7c211e4df
Remove most of ErrorFactories [DPP-606] (#13146)
changelog_begin
changelog_end
2022-03-07 16:15:21 +01:00
pbatko-da
15aa300fba
Use DamlContextualizedErrorLogger.forTesting (#13137)
changelog_begin
changelog_end
2022-03-04 12:24:08 +01:00
pbatko-da
af27f5d926
Make ErrorFactories into an object in preparation for its removal. [DPP-606] (#13143)
changelog_begin
changelog_end
2022-03-04 09:35:38 +01:00
fabiotudone-da
095a017a31
Grant driver-specific API test tools access to KV offsets and ParticipantTestContext (#13023)
CHANGELOG_BEGIN
CHANGELOG_END
2022-02-22 16:59:18 +01:00
Remy
ac3d9a4647
LF: Add LoggingContext to Speedy Machine (#12976)
This advances the state of #12208

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-21 13:34:46 +01:00
pbatko-da
98d5bd5f53
[User management] Add submission-id to the logs for mutating user management calls [DPP-887] (#12955)
changelog_begin
changelog_end
2022-02-17 14:29:58 +01:00
Remy
380dfa3502
LF: Add cause to internal error (#12965)
When an internal error is due to an unexpected exception, we wrap the
exception into the InternalError.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-16 11:06:10 +00:00
nicu-da
55fef9cf6a
[ledger-api] - Rename max deduplication time to max deduplication duration [KVL-1311] (#12939)
changelog_begin
ledger-api - rename ledger configuration max_deduplication_time to max_deduplication_duration
changelog_end
2022-02-15 21:14:26 +01:00
Kamil Bozek
dfd38186fe
Remove pre-1.18 error codes [DPP-773] (#12841)
* Remove legacy error codes

CHANGELOG_BEGIN
CHANGELOG_END

* Remove ValidatorFixture

* Removed redundant helper function

* Remove redundant object

* Rebased.

run-full-compat: true

* Rebased.

run-full-compat: true
2022-02-15 15:28:36 +01:00
tudor-da
307931d5b4
Extracted kvutils Config to //leger/ledger-runner-common [DPP-865] (#12889)
* Extracted kvutils Config to //leger/ledger-runner-config

changelog_begin
changelog_end

* Sandbox-on-X logback.xml does not depend on the kvutils/app one

* Remove unused dependencies

* Add copyright header back

* Remove dependency of SoX on kvutils
2022-02-14 13:52:06 +00:00
tudor-da
dd6dab6e6b
[Sandbox-on-X] Address various TODOs (#12823)
* Removed no-action TODOs

changelog_begin
changelog_end

* Assert self-service error codes in SequenceSpec

* Move InternallyInconsistentKeys and InternallyDuplicateKeys to LedgerApiErrors.WriteServiceRejections

* Restructured ledger bridge metrics
* Buffers before each processing stage

* Do not log the failure stacktrace on ledgerId lookup

* Addressed review comments

* Buffer size before prepare submission is 128
2022-02-11 17:03:41 +00:00
mziolekda
42ff83d456
Remove daml on sql artifacts and docs (#12870)
* stop publishing daml on sql aka Daml Driver for PotgreSQL

CHANGELOG_BEGIN
Stop publishing Daml Driver for PostgreSQL
CHANGELOG_END
2022-02-10 18:01:18 +00:00
Kamil Bozek
77401d452c
Remove the possibility to turn on legacy error codes [DPP-773] (#12721)
* Remove the possibility to turn on legacy error codes

CHANGELOG_BEGIN
- Switching to the legacy error codes is not possible.
CHANGELOG_END

* Revert breaking experimental features proto definitions

* Deprecate legacy error codes in proto definitions

* Proper todo naming

* Adjust compatibility tests

* Fix unintentional paste
2022-02-10 10:09:02 +01:00
tudor-da
c9dcd1c493
Update kvutils Config with missing flags options from SandboxConfig [DPP-864] (#12629)
* Update kvutils Config and SandboxConfig
* Remove `eager-package-loading` and `scenario` from SandboxConfig
* Add `profile-dir` and `stackTraces` to kvutils Config
* Configurable authService in kvutils

changelog_begin
changelog_end

* Addressed Samir's review comments

* Add back Maximum supported deduplication duration enforcement of 1 hour
2022-02-07 20:13:04 +00:00
pbatko-da
fa0062303d
[User management] Use JVM timestamps [DPP-798] (#12662)
I.e. calculate number of microseconds since Unix epoch in JVM rather than separately in each supported database.

changelog_begin
changelog_end
2022-02-07 15:22:28 +01:00
pbatko-da
6926e9adb4
Deprecate user identity service [DPP-862] (#12694)
changelog_begin
Ledger API Specification: Ledger Identity service is now deprecated.
changelog_end
2022-02-04 10:32:09 +01:00
pbatko-da
7137b46543
[User management] Use pagination for listing users [DPP-840] (#12610)
CHANGELOG_BEGIN
Ledger API Specification: UserManagementService.ListUsers is now using pagination
CHANGELOG_END
2022-02-03 20:32:50 +01:00
Marton Nagy
afdde77df8
Fix indexer configuration propagation for sandbox-on-x (#12740)
changelog_begin
changelog_end
2022-02-03 16:41:57 +01:00
Victor Peter Rouven Müller
aa45f48798
Upgrade to Scala 2.13.8 (#12506)
* Upgrade to Scala 2.13.8

changelog_begin
changelog_end

* Update hash for scala in nixpkgs

* update more hashes for scala upgrade

* Fix most warnings etc.

* Fix remaining errors etc.

* Fix formatting

* Resolve last errors hopefully

* Fix ledger api common build file

* Combine imports & revert accidental change that broke the CI run

* Rename exporting vals to scriptExport & minimize diff

* Remove more wrong changes

* moved warning around
2022-02-03 09:05:31 +00:00
Samir Talwar
496bc4e45b
kvutils: Simplify CommandDeduplication somewhat, and clarify error messages. (#12708)
* kvutils: Clarify handling of all deduplication period cases.

* kvutils: Simplify CommandDeduplication somewhat.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-02 10:56:24 +00:00
tudor-da
5390505627
Remove participant-side command deduplication [DPP-848] (#12677)
* Remove participant-side command deduplication

changelog_begin
changelog_end

* Addressed review comments
2022-02-01 20:50:25 +00:00
pbatko-da
c72c27c967
[User management] Terminate ongoing streams when user state has changed [DPP-830] (#12437)
CHANGELOG_BEGIN
Ledger API Specification: When using user management based authorization streams will now get aborted on authenticated user's rights change.
CHANGELOG_END
2022-01-28 16:44:39 +01:00
pbatko-da
4ec336dd6a
[User management] Enforce 1k user rights limit [DPP-833] (#12558)
CHANGELOG_BEGIN
Ledger API Specification: Maximum number of user rights per user is now limited to 1000 and is added to UserManagementFeature in VersionService. getLedgerApiVersion endpoint.
CHANGELOG_END
2022-01-27 22:12:34 +01:00
Remy
5cdf9b95fb
Remove some unecessary implictly with Factory (#12559)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-26 13:28:26 +00:00
Stefano Baghino
f1cd4b1c7c
Remove dependencies on compatibility libraries (#12548)
Continues the work started in https://github.com/digital-asset/daml/pull/12543

These libraries were only needed to transition from Scala 2.12 to 2.13
and are no longer useful as all the necessary items are now available
in Scala 2.13.

changelog_begin
changelog_end
2022-01-24 18:04:07 +00:00
Stefano Baghino
aec3390904
Replace silencer plugin with built-in warning configuration (#12543)
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.

[Here][1] is s a blog post by the Scala team about it.

Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.

Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.

changelog_begin
changelog_end

[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
2022-01-24 15:01:35 +00:00