Commit Graph

2656 Commits

Author SHA1 Message Date
pbatko-da
7cd677e367
[ETQ-Testing] Prepare Benchtool for extending it to support huge party filters [DPP-1079] (#14246)
This a preparation before the adding support for huge party filters.

Some of the changes:
1. Scan for existing parties even if when there is not submission step configured so that we can enrich stream configs fully.
NOTE: ConfigEnricher will now never fail on missing party.
2. Extract party selection for contract creation to a separate class.
3. Expose `commandGenerationParallelism` and a couple of randomness providers to allow for reproducible party allotment to contracts - useful in tests.
4. Extract `benchtoolFixture` to reduce boilerplate in IT tests


changelog_begin
changelog_end
2022-06-24 09:33:05 +00:00
Simon Maxen
6a1e3afb71
Switch rate limiting to using self service error codes [DPP-1000] (#14216)
* Switch rate limiting to using self service error codes

changelog_begin
Rate limiting updated to use self service error codes.
changelog_end
2022-06-21 16:49:22 +01:00
Sergey Kisel
4cc98fe715
Fix compilation problem after PR merge (#14237)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-21 11:36:40 +00:00
Sergey Kisel
6a80a1a82f
Removal of sandbox-classic module and binary (#14201)
* Removal of sandbox-classic module and binary

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-21 11:09:51 +02:00
pbatko-da
aa8ad32304
[Benchtool] Package Benchtool related Daml modules separately from other modules [DPP-1089] (#14231)
Recent changes in //ledger/test-common in Daml modules changed the fully qualified names of templates used by Benchtool, even though they hadn't been changed directly.

This made it impossible to filter by template name in a recently created LR IndexDB snapshot.

To avoid spurious changes in fully qualified template names (from Benchtool perspective) we are now packaging Benchtool related Daml modules separately from other modules

changelog_begin
changelog_end
2022-06-21 10:38:15 +02:00
pbatko-da
c575d598ff
[ETQ-TESTING] Fix slow flat transactions stream [DPP-1090] (#14187)
changelog_begin
changelog_end
2022-06-21 09:15:08 +02:00
pbatko-da
13d7389aa7
[Benchtool] Allow for setting a timeout for every stream [DPP-1088] (#14226)
changelog_begin
changelog_end
2022-06-20 15:37:43 +02:00
Remy
5b139bea60
add regression test for #14171 in conformance test (#14212)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-20 07:41:50 +00:00
azure-pipelines[bot]
253b0b186c
bump canton to 20220617 (#14218)
* bump canton to 20220617

CHANGELOG_BEGIN
CHANGELOG_END

* activate conformance test for regression test for exceptions

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2022-06-19 23:10:22 +00:00
Remy
2ddd744a91
[Engine] fix and test #14175 (#14206)
follow up of #14175

this is part of #14171

CHANGELOG_BEGIN
CHANEGELOG_END
2022-06-16 22:45:50 +02:00
nickchapman-da
44a14d06c9
Regression test for rollback create bug (#14197)
* Regression test for issue 14148

changelog_begin
changelog_end

* update compatability
2022-06-16 14:58:19 +00:00
Remy
2d5094160e
[Engine] clean key uniqueness mode (#14194)
This is a follow up of #14175 and #14176.  

Basically
-  we kill the `On` mode which is not used anymore.
- we factorize the visibility check.
- we rename `LocalContractNotVisible` into `ContractNotVisible`

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-16 12:04:45 +00:00
Moisés Ackerman
78cc57b243
Add failing daml-test-file (#14176)
fix #14171

CHANGELOG_BEGIN

* [Bugfix] fix failing by-key ops after roll-backed archive and create
  See https://github.com/digital-asset/daml/issues/14171

CHANGELOG_END
2022-06-15 20:00:04 +00:00
Sergey Kisel
f05acf217f
Expose createDataSourceProperties so defaults can be overriden in the downstream code. (#14184)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-15 09:14:25 +00:00
Simon Maxen
4d4c5ebb36
Rate limit when heap usage exceeds limit [DPP-997] (#14109)
* Rate limit when heap usage exceeds limit

changelog_begin
changelog_end
2022-06-15 09:00:10 +01:00
mziolekda
9c5c72a2fd
Update metrics documentation [DPP-1078] (#14174)
Correct metrics documentation shipped with the sdk
* Remove references to obsolete metrics for participant-side de-duplication
* Correct the names of the connection pool metrics
* Add a mention about internal service metrics
* Correct the name for the string interning metric
* Remove obsolete sandbox server role

CHANGELOG_BEGIN
Correct the name for the string interning metric from daml_index_db_loadStringInterningEntries to daml_index_db_load_string_interning_entries
CHANGELOG_END
2022-06-14 18:46:56 +02:00
Stefano Baghino
676095ada6
Fix typo in strong seeding initialization logging (#14172)
changelog_begin
changelog_end
2022-06-14 08:05:02 +00:00
Matthias Schmalz
865b882fdf
Documented why conversion methods are public (#14155)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-14 08:35:52 +02:00
Moritz Kiefer
d233bfa4b4
Add missing activeness check when reading from globalKeyInputs (#14080)
* Add missing activeness check when reading from globalKeyInputs

changelog_begin

- Engine. Fix lookup by key. See issue Lookup archived contracts may succeed when combined with rollback exceptions #14107 for more details.

changelog_end
2022-06-13 19:27:08 +00:00
Matthias Schmalz
639f8be207
Make methods for deserializing GRPC-commands public (#14144)
Its helpful in Canton tests, if we can reuse the existing code that converts GRPC-commands to API commands.

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-13 09:51:25 +02:00
Sergey Kisel
a6d6b7c324
Remove AuthServiceConfig from StandaloneApiServer (#14133)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-09 19:39:04 +02:00
Sergey Kisel
433be2cb35
Fixing regression in perf tests introduced by sandbox-classic removal (#14124)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-09 19:30:50 +02:00
Simon Maxen
855457a656
Allow bench test parties to be looked up or created [DPP-1001] (#14139)
* Allow bench test parties to be looked up or created

changelog_begin
changelog_end
2022-06-09 13:55:28 +01:00
Nicu Reut
944cc3142a
Remove package restriction for min/max time methods [kvl-1430] (#14131) 2022-06-08 16:22:24 +00:00
tudor-da
fc47a8995c
[LLP] Implement the possibility to have a shared StringInterningView [DPP-1042] (#14062)
* Allow the possibility to have a shared StringInterningView

changelog_begin
changelog_end

* Solve TODO in RecoveringIndexerIntegrationSpec

* Addressed Marton's review comments

* Added thread-safety mentionsin string interning interfaces
2022-06-08 16:13:31 +02:00
Stephen Compall
4075389f92
remove inherited choices from Scala codegen, test interface/template IDs (#14113)
* inherited choices are gone, convert to interface
* use toInterface for conformance tests on interfaces

CHANGELOG_BEGIN
- [Scala codegen] Interface choices can no longer be invoked directly on
  template IDs, or via ``createAnd`` or ``key`` directly.  Instead, use
  ``toInterface[Ifn]`` before calling the relevant ``exercise*`` method.

  The resulting ledger commands now contain the correct interface ID
  rather than template ID for looking up the choice, but see #13993 for
  a caveat regarding create-and-exercise and exercise-by-key.
CHANGELOG_END
2022-06-08 08:37:56 -04:00
Sergey Kisel
1d8f195438
Sandbox Classic Removal in Tests (#13934)
CHANGELOG_BEGING
CHANGELOG_END
2022-06-07 19:54:52 +00:00
tudor-da
ce98be8627
Extract ContractStateCaches from MutableCacheBackedContractStore (#14058)
changelog_begin
changelog_end
2022-06-06 16:12:40 +00:00
Marton Nagy
a939f7d76e
Refactoring: cleanup appendonly leftovers (#14096)
* Renames appendonlydao test directory
* Inlines JdbcAppendOnlyTransactionInsertion

changelog_begin
changelog_end
2022-06-05 00:07:54 +02:00
Marton Nagy
ec4f9850ba
ACS ID fetching enhancements [DPP-1058] (#14027)
* Adds exponential ID page retrieval to aid low stream initialization
* Adds Working-Memory concept to calculate parameters to aid guaranteed memory footprint
* Switches to dynamic BatchN batching instead of static grouping to aid stream initialization

changelog_begin
changelog_end
2022-06-04 14:42:28 +00:00
Andreas Lochbihler
4cfffec87a
Extract a contract state machine for transaction interpretation and validation (#14044)
CHANGELOG_BEGIN
CHANGELOG_END
2022-06-02 09:39:37 +00:00
Marton Nagy
f5a2539d6f
Add globalKeyMapping to WriteService [DPP-1067] (#14026)
The field globalKeyMapping is recently added to
lf.transaction.Transaction.Metadata, and which is
computed by the Engine.
This field might be needed for processing by WriteService
implementors.
This PR adds this to the WriteService.

changelog_begin
Global contract key mapping is added to WriteService.
changelog_end
2022-06-01 10:08:18 +02:00
tudor-da
3c5a52b895
[LLP] Move ledger-api-transactions-buffer-max-size to Config from ParticipantConfig (#14009)
* Move `ledger-api-transactions-buffer-max-size` to Config from ParticipantConfig

changelog_begin
changelog_end

* Update ledger/ledger-runner-common/src/main/scala/com/daml/ledger/runner/common/CliConfig.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2022-06-01 05:51:55 +00:00
Simon Maxen
4095f6bf49
Limit service API DB threadpool [DPP-998] (#14017)
Limit service API DB threadpool

changelog_begin
changelog_end
2022-05-31 14:00:02 +01:00
tudor-da
4b296fb5cc
Address Sergey's comment on BatchinParallelIngestionPipeSpec (#14023)
changelog_begin
changelog_end
2022-05-31 08:28:04 +00:00
tudor-da
2cb4fba2a2
Fix BatchingParallelIngestionPipeSpec flaky test (#14022)
changelog_begin
changelog_end
2022-05-31 07:09:39 +00:00
Sergey Kisel
168f943ff0
Revise the config to use a hierarchical structure [DPP-1029] (#13629)
* Revise the config to use a hierarchical structure [DPP-1029]

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-30 18:02:53 +02:00
Nicu Reut
2e1cc5c61b
[KV] - Replace RESOURCE_EXHAUSTED KV error with RESOURCE_OVERLOADED [kvl-1290] (#14014) 2022-05-30 14:37:17 +00:00
pbatko-da
c52b3bad79
Benchtool: Generate consuming events with non-stakeholder informees [DPP-1049] (#13988)
Non-stakeholder informees for consuming events are obtained by exercising a consuming choice on Foo1, Foo2 or Foo3 contract as a sub-action of divulgers newly introducedDivulgeConsumingExercise choice.

changelog_begin
changelog_end
2022-05-30 12:30:49 +01:00
tudor-da
0cda5d5c5a
Fix Sandbox-on-X error messages for inconsistent/duplicate key conflicts (#14008)
changelog_begin
changelog_end
2022-05-30 10:24:40 +02:00
pbatko-da
0de400680b
Benchtool: cancel streams when enough number of items has been seen [DPP-1051] (#13985)
Implementing it for all stream benchmarks:
 - transactions flat
 - transactions trees
 - active contracts
 - completions

changelog_begin
changelog_end
2022-05-27 15:28:11 +00:00
tudor-da
506af6c6eb
Introduce instrumented .buffered decorator for Akka Sources and Flows (#13997)
changelog_begin
changelog_end
2022-05-27 13:59:12 +00:00
pbatko-da
d463365492
Benchtool: Fix flaky test [DPP-1056] (#13998)
changelog_begin
changelog_end
2022-05-27 10:03:23 +00:00
Simon Maxen
40db23925d
Bump max api services queue size to 10,000 (#13996)
changelog_begin
changelog_end
2022-05-27 10:01:08 +00:00
pbatko-da
ef43df4849
Benchtool: Make non-consuming events visible to observer parties [DPP-1056] (#13987)
By making contract observers choice observer on the non-consuming choice (on Foo1, Foo2 and Foo3 contracts)


changelog_begin
changelog_end
2022-05-26 19:45:16 +02:00
tudor-da
b8c19d4c6a
[LLP] Akka optimizations [DPP-1040] (#13880)
* Akka optimizations for read pipeline
* Run computations on dedicated threadpools instead of the Akka dispatcher

changelog_begin
changelog_end

* Address Martino's comments

* Addressed Marton's review comments
2022-05-26 17:33:02 +00:00
Simon Maxen
374c294150
Allow the bench tool to either submitAndWait or submit [DPP-1062] (#13986)
* Make bench tool submission waiting optional [DPP-999]

changelog_begin
changelog_end
2022-05-26 13:31:50 +01:00
Remy
12a488f6a4
LF: Expose the keys requested during interpretation (#13983)
CHANGELOG_BEGIN
CHANGELOG_END
2022-05-25 19:03:35 +02:00
Simon Maxen
18e9473196
Rate limit [DPP-999] (#13956)
* RateLimitingInterceptor integrated into GrpcServer

changelog_begin
changelog_end
2022-05-25 17:13:49 +01:00
tudor-da
81f7e309db
Ledger API stream buffers moved in IndexServiceImpl (#13972)
changelog_begin
changelog_end
2022-05-25 11:24:09 +02:00