Commit Graph

2835 Commits

Author SHA1 Message Date
pbatko-da
bbaaac3058
[DPP-1278] Handle almost all the remaining 'um-for-hub' todos (#15261)
* InMemUserMngtImpl: Introducece custom in-mem user repr to better handle resource versions. Fix not updating isDeactivated property
* Generalize concurrent change control tests for users and party records
* Move backend files such that related localstore files are separated from other files
* Remove unneeded nesting in ParticipantParty object
* Document rights in revoke/grant requests as optional. More conformance tests for users rights

changelog_begin
changelog_end
2022-10-19 10:41:10 +02:00
atriantafyllos-da
e5c418a913
Bisected daml.execution.cache.register_update [DPP-1248]. (#15274)
CHANGELOG_BEGIN
* Removed daml_execution_cache_register_update metric
* Added:
  - daml_execution_cache_contract_state_register_update
  - daml_execution_cache_key_state_register_update
CHANGELOG_END
2022-10-18 17:33:13 +00:00
pbatko-da
ddfa7c1d9d
[DPP-1278] Do not test the exact matching of error messages in ObjectMeta related conformance tests (#15277)
changelog_begin
changelog_end
2022-10-18 17:41:24 +02:00
Gerolf Seitz
1454497f3a
Upgrade dependencies to their latest stable release (#15216)
The outdated dependencies where determined by running
`bazel run @maven//:outdated`.

Not all dependencies are brought up to the latest version available on
Maven Central, for the following reasons:

Nobody dares to touch DB:
  com.oracle.database.jdbc:ojdbc8 [19.14.0.0 -> 21.7.0.0]
  com.oracle.database.jdbc.debug:ojdbc8_g [19.14.0.0 -> 21.7.0.0]
  com.h2database:h2 [2.1.210 -> 2.1.214]
  org.flywaydb:flyway-core [8.4.1 -> 9.4.0]

License issues beyond version 2.6.x:
  com.typesafe.akka:akka-actor_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-actor-testkit-typed_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-actor-typed_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-http_2.13 [10.2.8 -> 10.4.0-M1]
  com.typesafe.akka:akka-http-spray-json_2.13 [10.2.8 -> 10.4.0-M1]
  com.typesafe.akka:akka-http-testkit_2.13 [10.2.8 -> 10.4.0-M1]
  com.typesafe.akka:akka-slf4j_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-stream_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-stream-testkit_2.13 [2.6.18 -> 2.7.0-M3]
  com.typesafe.akka:akka-testkit_2.13 [2.6.18 -> 2.7.0-M3]

Not a stable release:
  com.chuusai:shapeless_2.13 [2.3.3 -> 2.4.0-M1]
  io.circe:circe-core_2.13 [0.14.3 -> 0.15.0-M1]
  io.circe:circe-generic_2.13 [0.14.3 -> 0.15.0-M1]
  io.circe:circe-parser_2.13 [0.14.3 -> 0.15.0-M1]
  org.scalameta:munit_2.13 [0.7.29 -> 1.0.0-M6]
  org.scalactic:scalactic_2.13 [3.2.14 -> 3.3.0-SNAP3]
  org.scalatest:scalatest_2.13 [3.2.14 -> 3.3.0-SNAP3]
  org.scalatestplus:scalacheck-1-15_2.13 [3.2.11.0 -> 3.3.0.0-SNAP3]
  org.scalatestplus:selenium-3-141_2.13 [3.2.10.0 -> 3.3.0.0-SNAP3]
  org.scalatestplus:testng-6-7_2.13 [3.2.10.0 -> 3.3.0.0-SNAP3]
  org.tpolecat:doobie-core_2.13 [0.13.4 -> 1.0.0-RC2]
  org.tpolecat:doobie-hikari_2.13 [0.13.4 -> 1.0.0-RC2]
  org.tpolecat:doobie-postgres_2.13 [0.13.4 -> 1.0.0-RC2]
  com.fasterxml.jackson.core:jackson-core [2.13.4 -> 2.14.0-rc2]
  com.fasterxml.jackson.core:jackson-databind [2.13.4 -> 2.14.0-rc2]

Netty + boringssl + grpc + protobuf need be kept in sync, and the latest
versions also aren't stable releases:
  io.netty:netty-buffer [4.1.79.Final -> 5.0.0.Alpha2]
  io.netty:netty-codec-http2 [4.1.79.Final -> 5.0.0.Alpha2]
  io.netty:netty-handler [4.1.79.Final -> 5.0.0.Alpha2]
  io.netty:netty-handler-proxy [4.1.79.Final -> 5.0.0.Alpha2]
  io.netty:netty-resolver [4.1.79.Final -> 5.0.0.Alpha2]
  com.google.protobuf:protobuf-java [3.21.7 -> 4.0.0-rc-2]

Scalaz changed a few annoying things in 7.3, so let's stick to 7.2:
  org.scalaz:scalaz-core_2.13 [7.2.34 -> 7.4.0-M12]
  org.scalaz:scalaz-scalacheck-binding_2.13 [7.2.34-scalacheck-1.15 -> 7.4.0-M12]
  org.scalacheck:scalacheck_2.13 [1.15.4 -> 1.17.0]

Determined by the rules_scala:
  org.scala-lang:scala-library [2.13.8 -> 2.13.10]

CHANGELOG_BEGIN
Upgraded to the latest stable version of the following JVM dependencies:

ch.qos.logback:logback-classic:1.4.4
ch.qos.logback:logback-core:1.4.4
com.auth0:java-jwt:4.1.0
com.auth0:jwks-rsa:0.21.2
com.fasterxml.jackson.core:jackson-core:2.13.4
com.fasterxml.jackson.core:jackson-databind:2.13.4
com.github.ben-manes.caffeine:caffeine:3.1.1
com.github.pathikrit:better-files_2.13:3.9.1
com.github.pureconfig:pureconfig-core_2.13:0.17.1
com.github.pureconfig:pureconfig-generic-base_2.13:0.17.1"
com.github.pureconfig:pureconfig-generic_2.13:0.17.1
com.github.pureconfig:pureconfig_2.13:0.17.1
com.github.scopt:scopt_2.13:4.1.0
com.google.code.gson:gson:2.9.1
com.lihaoyi:pprint_2._13:0.8.0"
com.lihaoyi:sourcecode_2.13:0.3.0
com.sparkjava:spark-core:2.9.4
com.squareup:javapoet:1.13.0
com.storm-enroute:scalameter-core_2.13:0.21
com.storm-enroute:scalameter_2.13:0.21
com.thesamet.scalapb:scalapb-json4s_2.13:0.12.0
com.typesafe.scala-logging:scala-logging_2.13:3.9.5
com.zaxxer:HikariCP:5.0.1
commons-io:commons-io:2.11.0
eu.rekawek.toxiproxy:toxiproxy-java:2.1.7
io.circe:circe-core_2.13:0.14.3
io.circe:circe-generic-extras_2.13:0.14.2
io.circe:circe-generic_2.13:0.14.3
io.circe:circe-parser_2.13:0.14.3
io.circe:circe-yaml_2.13:0.14.1
io.github.paoloboni:spray-json-derived-codecs_2.13:2.3.9
io.reactivex.rxjava2:rxjava:2.2.21
io.spray:spray-json_2.13:1.3.6
javax.annotation:javax.annotation-api:1.3.2
javax.ws.rs:javax.ws.rs-api:2.1.1
junit:junit-dep:4.11
junit:junit:4.13.2
net.logstash.logback:logstash-logback-encoder:7.2
org.apache.commons:commons-lang3:3.12.0
org.apache.commons:commons-text:1.10.0
org.awaitility:awaitility:4.2.0
org.checkerframework:checker-qual:3.26.0
org.checkerframework:checker:3.26.0
org.codehaus.janino:janino:3.1.8
org.freemarker:freemarker-gae:2.3.31
org.jline:jline-reader:3.21.0
org.jline:jline:3.21.0
org.junit.jupiter:junit-jupiter-api:5.9.1
org.junit.jupiter:junit-jupiter-engine:5.9.1
org.junit.platform:junit-platform-engine:1.9.1
org.junit.platform:junit-platform-runner:1.9.1
org.mockito:mockito-core:4.6.1
org.mockito:mockito-inline:4.6.1
org.mockito:mockito-scala_2.13:1.17.12
org.pcollections:pcollections:3.2.0
org.playframework.anorm:anorm-akka_2.13:2.7.0
org.playframework.anorm:anorm_2.13:2.7.0
org.postgresql:postgresql:42.5.0
org.reactivestreams:reactive-streams-tck:1.0.4
org.reactivestreams:reactive-streams:1.0.4
org.reflections:reflections:0.10.2
org.sangria-graphql:sangria-ast_2.13:3.4.0
org.sangria-graphql:sangria-core_2.13:3.4.0
org.sangria-graphql:sangria-derivation_2.13:3.4.0
org.sangria-graphql:sangria-parser_2.13:3.4.0
org.sangria-graphql:sangria-spray-json_2.13:1.0.3
org.scala-lang.modules:scala-parallel-collections_2.13:1.0.4
org.scalactic:scalactic_2.13:3.2.10
org.scalameta:munit_2.13:0.7.29
org.scalatest:scalatest_2.13:3.2.10
org.scalatestplus:scalacheck-1-15_2.13:3.2.10.0
org.scalatestplus:selenium-3-141_2.13:3.2.10.0
org.scalatestplus:testng-6-7_2.13:3.2.10.0
org.scalaz:scalaz-core_2.13:7.2.34
org.scalaz:scalaz-scalacheck-binding_2.13:7.2.34-scalacheck-1.15
org.seleniumhq.selenium:selenium-java:4.5.0
org.slf4j:slf4j-api:2.0.3
org.slf4j:slf4j-simple:2.0.3
org.testng:testng:6.7
org.typelevel:cats-core_2.13:2.8.0
org.typelevel:cats-kernel_2.13:2.8.0
org.typelevel:cats-laws_2.13:2.8.0
org.typelevel:paiges-core_2.13:0.4.2
org.wartremover:wartremover_2.13.8:3.0.6
org.xerial:sqlite-jdbc:3.39.3.0
CHANGELOG_END
2022-10-18 08:50:23 +02:00
pbatko-da
61e85b19a2
[DPP-1278] Remove custom daml-script UserManagementStore (#15266)
changelog_begin
changelog_end
2022-10-17 20:58:41 +02:00
pbatko-da
04a5d2a12a
[DPP-1278] Address a portion of um-for-hub todos left in the codebase (#15255)
changelog_begin
changelog_end
2022-10-17 17:51:58 +00:00
pbatko-da
0d88435de1
Do not catch fatal errors in error intercpetor and hikari connection provider (#15263)
changelog_begin
changelog_end
2022-10-17 16:39:08 +00:00
pbatko-da
f0c2275726
[DPP-1278] User/party management: do not assert on detailed error messages in conformance tests as they should not be subject to compatibility checking (#15251)
changelog_begin
changelog_end

By doing this change we are loosing some detailed test coverage, but preventing future spurious compatibility tests failures.
2022-10-17 15:29:47 +00:00
pbatko-da
971e4f09a4
[DPP-1278] Treat empty string displayName as lack of displayName on DB result parsing (#15238)
changelog_begin
changelog_end
2022-10-17 14:48:05 +02:00
mziolekda
4f08bedc17
Print all kinds of problems in hikari connection checker (#15248)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-17 11:13:41 +02:00
atriantafyllos-da
884ee930d4
Tag all the metrics in the daml sdk code (#15202)
Tag metrics in daml [DPP-1225].

CHANGELOG_BEGIN
CHANGELOG_END
2022-10-13 15:51:55 +00:00
pbatko-da
e766b5d333
[DPP-1219] Organize packages in participant-local-store Bazel package (#15206)
changelog_begin
changelog_end
2022-10-13 09:58:09 +02:00
tudor-da
500a1f2ab2
Clean-up unused methods in MutableCacheBackedContractStore (#15213)
changelog_begin
changelog_end
2022-10-12 07:55:43 +00:00
atriantafyllos-da
ac8618aaed
Removed unused cache metrics [DPP-1246]. (#15149)
Removed unused cache metrics [DPP-1246].

CHANGELOG_BEGIN
The following metrics were removed from the metrics relevant to caches:
  - load_successes
  - load_failures
  - load_total_time
CHANGELOG_END
2022-10-11 16:39:56 +02:00
Marton Nagy
e5b173a9c4
Fix flaky test (#15205)
* Fix CommandServiceBackpressureIT by increasing load
* Make stage buffer size configurable (allows to limit the drain
effect of the heavily buffered pipeline)

CHANGELOG_BEGIN
CHANGELOG_END
2022-10-11 15:20:21 +02:00
pbatko-da
80df42bfcb
[DPP-1239] Unify metadata related tests for party and user management services (#15104)
changelog_begin
changelog_end
2022-10-11 10:01:48 +02:00
Nicu Reut
2f453a145c
Extract the testing metrics instance [PLEN-4] (#15192) 2022-10-10 18:52:49 +00:00
Simon Maxen
c5cd904dba
Exclude zero counts from metering report (#15169)
changelog_begin
changelog_end
2022-10-10 13:45:05 +01:00
Simon Maxen
b9abbfdcbf
Allow the rate limiting interceptor to be passed to api services (#15190)
changelog_begin
changelog_end
2022-10-10 11:24:01 +01:00
atriantafyllos-da
fd38b652ec
Update caffeine to 3.0.5. (#15176)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-06 17:12:39 +02:00
Sergey Kisel
de7a946d2e
Additional security tests for Ledger API [DPP-1256] (#15166)
* Additional security tests for Ledger API [DPP-1256]

CHANGELOG_BEGIN
CHANGELOG_END

* Additional security tests for Ledger API [DPP-1256]

CHANGELOG_BEGIN
CHANGELOG_END
2022-10-06 10:37:58 +02:00
Marton Nagy
0f5c3c3305
Contract Store Refactoring (#15075)
* Segregates ContractStore (pulls out MaximumLedgerTimeService)
* Renames lookupContractForValidation to lookupContractStateWithoutDivulgence, and adds the contractInstance to the results
* Adds implementation for MaximumLedgerTime computation depending on lookupContractStateWithoutDivulgence
* Adds unit tests for the implementation
* Adapts/fixes other tests
* Removes original MaximumLedgerTime implementation from MutableCacheBackedContractStore

CHANGELOG_BEGIN
CHANGELOG_END
2022-10-05 19:05:59 +00:00
atriantafyllos-da
b3a44cb3ee
Removed MeteredLedgerDao [DPP-1242]. (#15148)
CHANGELOG_BEGIN
The following metrics were removed:
  - daml.index.db.store_party_entry
  - daml.index.db.store_package_entry
  - daml.index.db.store_ledger_entry_combined
  - daml.index.db.store_rejection
  - daml.index.db.store_configuration_entry
  - daml.index.db.lookup_ledger_id
  - daml.index.db.lookup_participant_id
  - daml.index.db.lookup_ledger_end
  - daml.index.db.lookup_ledger_configuration
  - daml.index.db.get_parties
  - daml.index.db.list_known_parties
  - daml.index.db.list_lf_packages
  - daml.index.db.get_lf_archive
  - daml.index.db.prune
CHANGELOG_END
2022-10-05 15:47:32 +02:00
Sergey Kisel
759ce0b8e0
Additional security tests for Ledger API [DPP-1256] (#15161)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-05 08:47:18 +02:00
tudor-da
ae037ab5fd
Fail Akka sources with distinct throwables [DPP-1241] (#15084)
* On Dispatcher cancel, fail sources with distinct throwables

changelog_begin
Fixes an issue where failing multiple sources with the same StatusRuntimException
leads to an unhandled error in the gRPC serialization layer. This issue was
caused by serialization actually mutating the non-thread-safe io.grpc.Metadata
contained in the StatusRuntimeException.
changelog_end

* Fix ApiVersionService internalError

* Log error on ServerCall.close() errors

* Address Sergey's review comments

* Addressed more review comments

* Accessor method for call.close to bypass IllegalAccessErrors

* Use unique kill switches per stream in generated Akka Grpc streaming services

This change has two benefits:
- The error is logged only on an actual stream termination
- The throwable is not shared between streams, avoiding concurrent Metadata access errors

* Addressed Marton's review comments
2022-10-04 11:39:38 +00:00
pbatko-da
a55ccc384e
[DPP-1219] Move local stores for participant users and parties to a separate bazel target (#15108)
changelog_begin
changelog_end
2022-10-03 17:26:28 +02:00
atriantafyllos-da
507f18ad20
Dpp 1222 move metric tagging classes to daml sdk (#15067)
Moved canton's tagging metric class [DPP-1222].

CHANGELOG_BEGIN
CHANGELOG_END
2022-10-03 14:23:07 +02:00
Sergey Kisel
7787f5e1fe
InterfaceSubscriptionsIT Test for include_create_arguments_blob (#15107)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-03 10:06:47 +02:00
Remy
604e89cc77
Enable DeeplyNestedValueIT for canton (#15140)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-03 08:19:14 +02:00
mziolekda
10f874f465
Make test tool adopt to the level of user-management support [DPP-1250] (#15135)
* Make test tool adopt to the stage of user-management support

CHANGELOG_BEGIN
CHANGELOG_END

* Fix broken test

* Fix post review
2022-09-30 15:45:11 +02:00
tudor-da
0f59026a67
[ED] Clean-up [DPP-1210] (#15127)
* [ED Clean-up] Extract validateTimestamp to FieldValidations and improve error messages

changelog_begin
changelog_end

* Permanently enable ExplicitDisclosureIT:EDMetadata

* Cleanup ExplicitDisclosureIT
2022-09-30 14:34:59 +02:00
Simon Maxen
0654f9978c
Add libs-scala/struct-json (#15090)
changelog_begin
changelog_end
2022-09-29 16:22:01 +01:00
pbatko-da
f2a1d6c8ca
Conformance tests: Ensure unique test identifiers per suite rather than globally (#15118)
changelog_begin
changelog_end
2022-09-28 15:09:02 +02:00
pbatko-da
7652944a9c
Improve wording of the conveyance message in error codes docs (#15043)
changelog_begin
changelog_end
2022-09-28 08:55:12 +00:00
Hubert Slojewski
9102247857
Prevent ledger-api-test-tool's Reporter from throwing misleading NPEs [KVL-1580] (#15109)
CHANGELOG_BEGIN
CHANGELOG_END
2022-09-28 10:51:32 +02:00
tudor-da
a434530656
Change logLevel to warn on missing update tracker key in StateCache (#15069)
changelog_begin
changelog_end
2022-09-28 06:13:55 +02:00
Sergey Kisel
9ce2830cd4
Contract Argument Blob API change for the Explicit Disclosure [DPP-1162] (#15012)
* Contract argument blob [DPP-1161].

CHANGELOG_BEGIN

[Ledger API]: Introduce an ability to expose contract arguments as blob if matched by the interface filter using `include_create_arguments_blob` flag in the ``InterfaceFilter``.
Introduce an ability to provide `DisclosedContract.arguments` as a blob instead of a typed `Record`.

CHANGELOG_END

Breaks-protobuf: true


Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
Co-authored-by: atriantafyllos-da <andreas.triantafyllos@digitalasset.com>
2022-09-27 14:30:47 +02:00
pbatko-da
ca81adeb3e
6 - Ledger API changes for UM/UP extensions for Hub [DPP-1211] (#14937)
changelog_begin

[Ledger API Specification]:
1. Add 'is_deactivated' attribute to participant users.
2. Introduce participant server's local metadata for parties and participant users consisting of:
 - a resource version for optional concurrent change control,
 - modifiable key-value based annotations.
3. Add experimental update RPCs for parties and participant users:
  - participant user's modifiable fields are: 'primary_party', 'is_deactivated', 'metadata.annotations',
  - party's modifiable fields are 'local_metadata.annotations'.
4. Discourage use of 'party_details.display_name' in favor of using party's metadata annotations.

changelog_end


Other notes:

- no-op updates are allowed,
- update paths are resource relative,
- as annotation deletion is by providing an key with the empty string value,
- as a corollary, only non-empty strings are valid annotation values.
2022-09-26 12:56:55 +00:00
Remy
6dd9a45e50
[Engine] small cleaning of LookupError (#15060)
* [Engine] small cleaning of LookupError

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-23 13:46:14 +00:00
tudor-da
e0dc3f8679
[ED] Enable explicit disclosure in conformance tests [DPP-1206] (#14974)
* Allow enabling explicit disclosure in Sandbox-on-X

changelog_begin
changelog_end

* Fix conformance tests

* Disable tests testing yet unsupported features

* Addressed Martino's review comments
2022-09-23 07:55:19 +00:00
tudor-da
5dca9bddb8
Fix flaky ConflictCheckWithCommittedSpec due to timeout (#15066)
changelog_begin
changelog_end
2022-09-22 16:31:42 +02:00
Sergey Kisel
118ac6e2fd
Rename unknown key (#15063)
CHANGELOG_BEGIN
CHANGELOG_END
2022-09-22 13:43:23 +02:00
tudor-da
298920a323
[ED] Forward contract metadata in Ledger API streams (#14906)
changelog_begin
The `CreatedEvent.metadata` field is populated for `created_at` and `contract_key_hash` (where applicable)
for active contracts and transactions payloads in Ledger API streams
changelog_end
2022-09-22 12:19:27 +02:00
mziolekda
41dc32a016
Remove reliance on implicit party allocation in CI tests [DPP-1234] (#15037)
* Remove reliance on implict party allocation in the authorization tests of the sandbox-on-x

CHANGELOG_BEGIN
CHANGELOG_END

* Remove the HOCON and cli config parameters

* Remove implicit party allocation from the Submission Service

* Remove support for implicit parties from the bridge

* Fix codegen reliance on implicit party allocation

* Fix daml script tests

* Fix trigger tests

* format security-evidence.md post rebase

* Fix json-api tests

* One more json-api test

* Fix haskell tests

* Corrections post-review

* Cosmetic improvements

* Fix race condition in party allocation handling
2022-09-22 10:20:53 +02:00
Carl Pulley
aa062cde42
Final step of explicit disclosure cached contract refactor
* Fixes #13897

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-21 18:12:55 +01:00
Sergey Kisel
ca92e45563
Tighten pure config validation [DPP-1205] (#15044)
* Tighten pure config validation [DPP-1205]

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-21 15:00:49 +02:00
Sergey Kisel
86cff50d1a
Ability to override participantId [DPP-1143] (#15029)
* Ability to override participantId [DPP-1143]

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2022-09-19 12:56:49 +00:00
Sergey Kisel
9006f841c5
Remove references to Sandbox-on-X from shared help [DPP-1204] (#15027)
CHANGELOG_BEGIN
CHANGELOG_END
2022-09-15 08:48:31 +00:00
mziolekda
161e430482
Handle exception in hikari connection health check [DPP-1232] (#14998)
* handle error using error code framework

CHANGELOG_BEGIN
CHANGELOG_END

* handle error using standard logger

* clean up

* sort imports

* mute recurrent errors
2022-09-14 09:27:20 +02:00
Sergey Kisel
4a1e9f347e
Reverting as fix propagated to canton [DPP-1220] (#15011)
CHANGELOG_BEGIN
CHANGELOG_END
2022-09-13 11:04:17 +02:00