Commit Graph

2594 Commits

Author SHA1 Message Date
pbatko-da
e1b7d120b4
Benchtool: Move observers to a test-lib target [DPP-979] (#13892)
changelog_begin
changelog_end
2022-05-17 14:40:31 +02:00
pbatko-da
e9091279c0
Benchtool: Improve assertion for non-stakeholder informees test [DPP-978] (#13894)
changelog_begin
changelog_end
2022-05-17 14:38:17 +02:00
Marton Nagy
fee168ca6e
Removes exclusion of canton conformance-test [DPP-1018] (#13888)
CHANGELOG_BEGIN
CHANGELOG_END
2022-05-17 09:47:55 +02:00
Marton Nagy
de6da71083
Minor readability fix: makes timeout related code more obvious (#13813)
CHANGELOG_BEGIN
CHANGELOG_END
2022-05-17 01:25:58 +02:00
Remy
6f6a3052a1
Repurpose templateId in ExerciseCommand and add interfaceId in ExerciseEvent (#13660)
part of #13653

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-16 15:30:07 +00:00
pbatko-da
aad0e05533
Benchtool: Improve payload size handling [DPP-1028] (#13838)
Previous approach was generating random bytes
and encoding them into UTF-8 strings.
This turned out to be problematic as not every
random byte sequence is valid in the UTF-8 charset, and
`java.lang.String#String(byte[], java.lang.String)` constructor's
documentation states that in such cases its behavior
is unspecified.

In practice the size of the byte array obtained from decoding the
generated string was almost twice the size of the orginal byte array.

Current approach limits the generated bytes to valid ASCII characters,
each of which takes exactly one byte in UTF-8 encoding.

changelog_begin
changelog_end
2022-05-16 15:27:33 +02:00
pbatko-da
724a0c1b1a
Benchtool: Non-stakeholder informees generation [DPP-978] (#13808)
We obtain non-empty non-stakeholder informees by performing an immediate divulgence of instances of Foo1, Foo2 or Foo3 templates through instances of a helper Divulger template.
The divulgence is controlled by 1) configuring the number of all divulgees to generated and 2) probabilistically picking a non-empty subset from all divulgees for each contract (similar to how observers are picked for each contract with certain probabilities). If the divulgees for a contract are non-empty a Foo1, Foo2 or Foo3 contract is created via a helper choice on a Divulger contract. Otherwise a standard create Foo1, Foo2 or Foo3 command is issued.

Each Divulger contract can divulge any number of Foo1, Foo2 or Foo3 contracts but it's limited to divulging to a fixed set of divulgees. In other words, there is a separate Divulger contract for each non-empty subset of all divulgees.
We expect the set of all divulgees to be very small in practice similar to how only a small number observers (currently at most 3) is currently configured for existing benchmarks. Thus, the set of all subsets is also small.
To be conservative, we allow the number of divulgees to be at most 5.

changelog_begin
changelog_end
2022-05-16 15:27:03 +02:00
pbatko-da
751e0f78a6
Benchtool: Separate command generation and submission. Also other clean-ups [DPP-978] (#13833)
This is setting ground for generating contracts with non-empty non-stakeholder informees in #13808

changelog_begin
changelog_end
2022-05-12 11:40:59 +02:00
Sergey Kisel
37b958beb1
Removal of sandbox-common-ce as redundant (#13824)
* Removal of `sandbox-common-ce` as redundant

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-10 13:59:52 +02:00
Nicu Reut
d886dd1a1a
Ensure we account for time model all success/failure in the conformance tests. (#13836)
- Wait for confirmation for all submissions when setting time model
- Do not ignore authorization errors, this is done so that we are consistent across tests (exclusions should be used if required)

changelog_begin
changelog_end
2022-05-10 09:19:00 +00:00
Nicu Reut
be275bb5ec
Upload dars sequentially for every participant defined [kvl-1369] (#13834) 2022-05-10 08:18:51 +00:00
tudor-da
2614646a35
[LLP] Mutable contract state cache uses Offset as cache_index (#13733)
changelog_begin
changelog_end
2022-05-09 11:23:34 +02:00
tudor-da
2115fbf55b
[LLP] Indexer latency optimizations [DPP-1014] (#13667)
* [LLP] Indexer latency optimizations

changelog_begin
changelog_end

* Addressed Robert's review comments

* Addressed Marton's review comments
2022-05-07 07:01:25 +02:00
Marton Nagy
5d8ebd7a08
Switch to asynchronous teardown at Dispatcher (#13631)
Previously a fire and forget release process
was implemented, which in corner cases could
result in exceptions, as API stream was still running after DbDispatcher was released.

This PR changes to waiting for all the streams to finish as releasing Dispatcher (with a timeout of 5 seconds).

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-05 21:15:58 +02:00
Sergey Kisel
50c0f233e8
Address review concerns on security test evidence. (#13794)
* Address review concerns on security test evidence.

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-05 14:12:05 +00:00
Nicu Reut
229a8d973a
conformance-tests: Account for min/max skew twice and scale duration for command dedup test [kvl-1426] (#13785) 2022-05-05 14:42:28 +02:00
tudor-da
ab6af91ff2
Fix slow lookupMaximumLedgerTime (#13761)
changelog_begin
changelog_end
2022-05-04 15:24:32 +02:00
Simon Maxen
38434547d1
Post mortem fixup of bench tool submission [DPP-1017] (#13786)
* Post mortem fixup of bench tool submission

changelog_begin
changelog_end

* Update with review comments
2022-05-04 12:58:33 +01:00
Robert Autenrieth
1b9e990511
DPP-925 Move package object (#13741)
* Move package object

changelog_begin
changelog_end

* More consistently use aliases
2022-05-03 21:53:21 +02:00
Simon Maxen
da0c458b8a
Add support for fibonacci contracts. (#13738)
changelog_begin
changelog_end
2022-05-03 16:16:56 +01:00
Marton Nagy
0ea140f51e
Fix random exercise node's childEventId order [DPP-1018] (#13740)
Fixes Index DB insertion order for childEventId.
Adds backwards compatibility treatment: recovering childEventId
order from the order of events at API stream rendering.
Extends TransactionServiceVisibilityIT with the ordering criterion.
Fixes compatibility tests with exclusion.
Adds documentation to event.proto.

CHANGELOG_BEGIN
Fixing Ledger API Bug: Exercise nodes in transaction trees
have child_event_ids out of order.
CHANGELOG_END
2022-05-03 16:02:25 +02:00
Robert Autenrieth
6bb59e6a0f
DPP-923 cleanup conversions (#13742)
* Clean up Conversions

changelog_begin
changelog_end

* Remove dead code
2022-05-03 15:26:29 +02:00
Simon Maxen
4df52d5c23
DPP-1087 - Make benchtool submitter extendable (#13721)
* Prefix command generator with Foo

* Extract CommandGenerator

* Update with review comments

changelog_begin
changelog_end

* Temporary name change
2022-05-03 13:27:25 +01:00
Robert Autenrieth
f2879f7bac
Move EventsTable.Entry (#13753)
changelog_begin
changelog_end
2022-05-03 14:20:56 +02:00
Robert Autenrieth
e691ffd6fc
Rename migration folders (#13752)
changelog_begin
changelog_end
2022-05-03 14:20:34 +02:00
Nicu Reut
a5b002eb32
ledger-api-test-tool - Adapt for concurrent conflicts [kvl-1369] (#13719) 2022-05-03 10:21:23 +00:00
tudor-da
83e6ad2696
Fix flaky RecoveringIndexerIntegrationSpec (#13777)
changelog_begin
changelog_end
2022-05-03 08:39:49 +00:00
Hubert Slojewski
281de9ac2e
Publish daml-lf encoder, parser and error-test-lib [KVL-1424] (#13730) 2022-04-29 13:59:19 +02:00
Robert Autenrieth
31015b68fc
DPP-748 Remove Java migrations (#13731)
* Remove Java migrations

changelog_begin
changelog_end

* Add test
2022-04-29 12:19:23 +02:00
Robert Autenrieth
d75cacf0ef
Rename store backend template files (#13714)
... if they are not template files

changelog_begin
changelog_end
2022-04-28 16:10:19 +02:00
Robert Autenrieth
3e898e04d5
DPP-926 Move SimpleSqlAsVectorOf (#13711)
* Move SimpleSqlAsVectorOf

changelog_begin
changelog_end

* Restrict visibility
2022-04-28 13:26:43 +02:00
Nicu Reut
a792de39a5
Update scalafmt to 3.5.1. (#13584)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-28 09:57:04 +00:00
Robert Autenrieth
2f625dff2c
Merge append-only tests (#13713)
changelog_begin
changelog_end
2022-04-28 10:16:00 +02:00
Nicu Reut
c2b44b0764
Remove override for definitive answer from KVError [kvl-1369] (#13728) 2022-04-28 07:55:34 +00:00
Robert Autenrieth
f0303b698e
Rename appendonlydao to dao (#13715)
* Rename appendonlydao to dao

changelog_begin
changelog_end

* fmt
2022-04-27 22:12:40 +02:00
Nicu Reut
04f991a340
[kvutils] - Add errors for post execution conflicts and out of time bounds [KVL-1369] (#13710) 2022-04-27 15:07:23 +00:00
Hubert Slojewski
61be3a5e39
Move kvutils out of the SDK [KVL-1245] (#13704) 2022-04-27 15:21:28 +02:00
pbatko-da
6d6c46596a
[ETQ] Add an integration test for Benchtool's command submission [DPP-972] (#13533)
changelog_begin
changelog_end
2022-04-26 19:19:33 +02:00
Hubert Slojewski
35b68962e8
Remove remaining kvutils dependencies [KVL-1245] (#13695)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 13:42:18 +02:00
Hubert Slojewski
d700fc9e76
Move kvutils test dar file lib to participant-state [KVL-1245] (#13696)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 13:42:04 +02:00
Robert Autenrieth
a88ae129b5
Use small buffers for tests (#13659)
* Use small buffers for tests

changelog_begin
changelog_end

* Update ledger/sandbox-on-x/BUILD.bazel

Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>

Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
2022-04-26 11:27:59 +02:00
Hubert Slojewski
0e9326f857
Publish daml-lf/snapshot proto [KVL-1245] (#13679)
* Publish daml-lf/snapshot proto [KVL-1245]

CHANGELOG_BEGIN
CHANGELOG_END

* fix snapshot_proto bazel target

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2022-04-26 07:59:12 +02:00
pbatko-da
d66045a169
[ETQ] Extend Benchtool to submit consuming and nonconsuming exercises [DPP-972] (#13492)
changelog_begin
changelog_end
2022-04-25 09:45:02 +02:00
pbatko-da
74b6e69f2f
Fix performance regression flat and tree transaction streams on Postgres (#13623)
Before Postgres ended up using sequential scans.
With added "order by" clauses Postgres uses index scans.

changelog_begin
changelog_end
2022-04-22 11:24:38 +02:00
Remy
3be994c4e8
LF: introduce interfaceId field in Exercise node (#13643)
.. to track where the choice is defined.

part of #13653

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-21 13:19:25 +00:00
Marton Nagy
94b3a5890e
Fix ACS id retrieval parallelism [DPP-986] (#13550)
This PR fixes an issue with the ACS ID page retrieval: if there is one
of the id page DB fetching is slow, all the rest of the ID fetching
lanes getting fetched, due to continuous demand.
We implemented a treatment to cap the increased number of ID pages
accumulated in memory, by failing the stream.

Switching to mapAsync in pullWorkerSource naturally will
stop demand, in case a single query takes too long.
This also removes the need for the queue size limitting treatment.

* Switches to mapAsync in pullWorkerSource
* Fixes unit test
* Removes queue limiting feature (as it is not necessary anymore)

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-20 15:20:19 +00:00
Nicu Reut
a37dd63675
Remove committer support for non pre-execution [kvl-734] (#13598)
CHANGELOG_BEGIN
[kvutils] - Pre-execution is the only supported execution mode
CHANGELOG_END
2022-04-20 09:52:53 +00:00
Nicu Reut
23feb9c513
Remove ledger-on-memory [kvl-1365] (#13599)
CHANGELOG_BEGIN
[kvutils] - remove ledger-on-memory
CHANGELOG_END
2022-04-20 10:55:51 +02:00
Hubert Slojewski
16a97e8715
Move KVErrors to participant-state-kv-errors [KVL-1371] (#13604)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-19 15:22:31 +02:00
Sergey Kisel
a6b3d5f581
Cleanup around configuration (#13597)
* Cleanup around configuration

CHANGELOG_BEGIN
CHANGELOG_END

* Removal of unused configuration parameters. Merge cohesive params.

* Removal of ParticipantIndexerConfig as redundant transformation layer.

* Extracting engine config values

* Separation of IndexConfiguration
2022-04-19 13:19:11 +02:00