Commit Graph

376 Commits

Author SHA1 Message Date
nicu-da
4b4fbf8c2a
[ledger-api-test-tool] - adapt the command deduplication conformance to use participant deduplication feature descriptor [kvl-1218] (#12222) 2022-01-04 07:10:48 -08:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Robert Autenrieth
d922a562a8
DPP-752 refactor event strategy (#12008)
* Reset string interning between tests

* Add StorageBackendEvents tests

* Extend LedgerDao ACS tests

* Remove dead code

changelog_begin
changelog_end

* Refactor EventStrategy

changelog_begin
changelog_end

* Remove dead code

* Refactor filterPartiesClause

* Avoid materializing a collection

* remove interning from arrayIntersectionNonEmptyClause

* Fix internedAllParties

* Add test for filtering witnesses

* Handle empty readers

* Add note about resetting string interning state

* Throw exceptions on unexpected empty input
2021-12-21 09:32:26 +01:00
nicu-da
5b142eba63
[ledger-api-test-tool] - use offset support feature descriptor [kvl-1218] (#12211) 2021-12-20 06:53:39 -08:00
nicu-da
e182fc5026
[ledger-api] - Move deduplication duration validation into the KV WriteService (#12151)
changelog_begin
ledger-api - Remove validation for duration deduplication periods, which was validating that the deduplication duration is smaller than the max deduplication duration. This is now up to the ledger to enforce.
changelog_end
2021-12-20 05:33:46 -08:00
nicu-da
453fd64d05
[ledger-api-test-tool] - Add ability to disable tests based on ledger features [kvl-1218] (#12185)
changelog_begin
changelog_end
2021-12-17 12:15:38 -08:00
fabiotudone-da
2940a7b5c2
[ledger-api-test-tool] - existing_submission_id and completion_offset assertions for ALREADY_EXISTS rejections in command dedup ledger API tests [KVL-1020] (#12140)
changelog_begin
changelog_end
2021-12-16 06:11:39 -08:00
Simon Meier
6c7e33cafe
Add Ledger API feature descriptor for user management. (#12095)
* Add Ledger API feature descriptor for user management.

Includes:
- enabling the UserManagementServiceIT tests by default; and skipping them for ledgers that do not support user management. The correct working of this is tested by the cross-version compatibility checks between the ledger-api-test-tool
- some refactoring wrt the parsing and representation of features the the ledger-api-test tool.

CHANGELOG_BEGIN
- [Ledger API]: introduce a feature descriptor for whether the ledger supports user management
CHANGELOG_END
2021-12-14 11:00:29 +01:00
Simon Meier
f223528bfb
Implement PoC of user management for Ledger API server (fix #12014) (#12063)
CHANGELOG_BEGIN
- [User Management]: add support for managing participant node users and authenticating
  requests as these users using standard JWT tokens.
CHANGELOG_END

Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
Co-authored-by: Adriaan Moors <90182053+adriaanm-da@users.noreply.github.com>
2021-12-13 17:58:30 +01:00
nicu-da
e1a2da0771
[ledger-api] - Add deduplication_offset as a valid deduplication period for commands [KVL-1172] (#12041)
changelog_begin
[ledger-api] - Add `deduplication_offset` as a valid type for the command deduplication period
changelog_end
2021-12-13 12:26:09 +00:00
Moritz Kiefer
9a01bb9b86
Fix filters in ExceptionRaceConditionIT (#12106)
* Fix filters in ExceptionRaceConditionIT

Unfortunately I messed up all the filters (probably got confused by
the very generic sounding names) and it resulted in us just asserting
on nothing.

I am a bit worried about the fragility of those tests. If you mistype
a choice name you run into the same issue. A potentially more robust
option might be to assert that the filters are non-empty or run them
with the assertion reversed and check that they fail but that seems
out of scope for this PR.

I did reverse all assertions manually and checked that the tests do
fail so they properly assert on what they should.

Very sorry for messing this up

changelog_begin
changelog_end

* remove unused defs

changelog_begin
changelog_end

* Inline filters

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end
2021-12-10 14:56:44 +00:00
mziolekda
96f0f4f63f
Introduce better TLS version tests (#11950)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-02 11:00:39 +01:00
tudor-da
abc141b57e
Increase pruning tests timeout (#11891)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 08:16:35 +00:00
tudor-da
8e31e8e6a8
[Self-service error codes] Enabled by default [DPP-691] (#11685)
* [Self-service error codes] Enabled by default
* Flag changed to `use-pre-1.18-error-codes` (disabled by default)

CHANGELOG_BEGIN
[Ledger API Specification] The Ledger API returns enriched error codes (see https://docs.daml.com/error-codes/self-service/index.html)
For backwards-compatibility, a new API flag `--use-pre-1.18-error-codes` is introduced for preserving the legacy behavior for
clients that want to migrate incrementally to the changed gRPC status code responses and error details format.
CHANGELOG_END

* Adapted HttpServiceIntegrationTest

* Renamed `Feature Flag` to `Configuration` in docs

* Fix Daml Script tests

changelog_begin
changelog_end

* Fix Repl functests

changelog_begin
changelog_end

* Fix haskell binding tests

changelog_begin
changelog_end

* Fix CommandClientIT test

* Fixed Sandbox and CommandServiceBackpressureIT tests
 Please enter the commit message for your changes. Lines starting

* Adapt //compiler/damlc/tests:repl-functests again

* Fix more tests and address Miklos' comments
* Flag name changed to `grpc-status-codes-compatibility-mode`

* Remove useless flags sandbox-classic

* Sandbox-classic tests fix for ContractKeysIT and ExceptionsIT
* Created 2 deprecated test suites that have the more generic assertions as returned
by the deprecated in-memory backend

* More fixes for CommandServiceIT
* Fixes compilation issue with the deprecated exceptionsIT class for Sandbox-classic in-memory

* Compatibility mode for old test tools

* Change flag name to `use-pre-1.18-error-codes`

* Apply suggestions from code review

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-11-22 12:26:48 +00:00
nicu-da
8f458d8d2a
[ledger-api] Include the completion offset in the command_service.proto responses [KVL-1171] (#11658)
CHANGELOG_BEGIN
[ledger-api] - Include the completion offset in the responses from the command_service.proto
CHANGELOG_END
2021-11-18 04:37:26 -08:00
nicu-da
dd88ba24c3
kvutils - Backwards looking command deduplication [KVL-1174] (#11634)
CHANGELOG_BEGIN
kvutils - The deduplication duration passed in the command is now used for command deduplication, and is no longer always overwritten with the max deduplication duration. The command deduplication duration can still be extended by the committer to account for time skews.
CHANGELOG_END
2021-11-18 02:25:31 -08:00
tudor-da
fbc436d70a
[DPP-739][Self-service error codes] Adapt QueueBackedTracker error codes (#11719)
* [Self-service error codes] Adapt QueueBasedTracker

CHANGELOG_BEGIN
CHANGELOG_END

* Added changed error codes to migration guide

* Fixed ErrorFactoriesSpec

* Update ledger/error/src/main/scala/com/daml/error/definitions/LedgerApiErrors.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: Pawel Batko <pawel.batko@digitalasset.com>
Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-16 22:21:29 +00:00
tudor-da
ec6d7cc3db
[Self-service error codes] Error code definitions revisiting [DPP-675] (#11686)
* Error definitions revisiting and hierarchical grouping refactoring

CHANGELOG_BEGIN
CHANGELOG_END

* Full hierarchical path for error grouping

* Addressed Pawel's review comments

* Apply suggestions from code review

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>

* Do not strip margin in error code explanations

* Revert changing LEDGER_ID_MISMATCH from NOT_FOUND to FAILED_PRECONDITION

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>
2021-11-15 21:28:13 +00:00
tudor-da
d6ca68dc23
[Self-service error codes] Adapt Sandbox classic rejection error codes (#11622)
* [Self-service error codes] Adapt Sandbox classic rejection error codes
* Unify common rejection error codes with KV
* Adapt conformance tests

CHANGELOG_BEGIN
CHANGELOG_END

* Fix JdbcLedgerDaoPostCommitValidationSpec

* Addressed first round of review comments

* Addressed Miklos' comments

* Revert camelCasing invalidLedgerTime arguments

* Moves DuplicateCommand to CommandRejections

* Addressed Gerolf's comments
2021-11-12 16:33:02 +00:00
tudor-da
2f4476c12a
[Self-service error codes] Fill in missing places with submission id for correlation id (#11593)
* [Self-service error codes] Fill in missing places with submission id for correlation id
* KVConsumption
* GrpcCommandSubmissionService

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments

* WriteService uses submission id as correlation id

* Populate the submission id in command submissions in ledger-api-test-tool

* Do not fail submission on missing submission id

* Fix KeyValueParticipantStateWriterSpec

* Addressed review comments
2021-11-12 09:08:06 +01:00
tudor-da
92dfcdeb24
[Self-service error codes] Use error validators in ApiPackageManagementService (#11575)
* Use error validators in ApiPackageManagementService

CHANGELOG_BEGIN
CHANGELOG_END

* Exclude PackageManagementServiceIT from compatibility tests after 1.17

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-09 21:54:26 +01:00
Hubert Slojewski
445031114f
Exclude/fix compatibility tests (#11529)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 19:50:49 +01:00
tudor-da
5e93df6b52
Full conformance test with self-service error codes on ledger-on-memory (#11524)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-04 09:36:06 +01:00
nicu-da
9bb1d6443c
[ledger-api] - Return FAILED_PRECONDITION gRPC status code for invalid deduplication duration [KVL-1170] (#11483)
CHANGELOG_BEGIN
[ledger-api] - Return FAILED_PRECONDITION gRPC status code, instead of INVALID_ARGUMENT, for an invalid command deduplication duration
CHANGELOG_END
2021-11-03 13:34:31 +00:00
pbatko-da
8a9f15b8de
[DPP-686][Self-service error codes] Removing default error factories (#11403)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-03 09:40:19 +01:00
tudor-da
3a5cb18e7f
[Self-service error codes] Adapt all grpc assertions to additionally take self-service error codes [DPP-680] (#11437)
* Adapt self-service error codes in conformance tests
* Assertions.assertGrpcError and assertGrpcErrorRegex assert self-service error codes based on feature descriptor
* Adapted all GRPC error assertions to expect self-service error codes as well
* Added TODO and exclusions list to conformance-test-self-service-error-codes in anticipation of some missing
self-service error codes implementation.

CHANGELOG_BEGIN
CHANGELOG_END

* Correct errorCode id assertion

* Fixed incorrect error assertion in CommandService

* Exclude entire KVCommandDeduplicationIT due to flakiness
* To be fixed in the subsequent PR
2021-11-01 20:43:47 +00:00
Remy
3098b70624
pretty print contract IDs properly (#11359)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-29 09:02:33 +02:00
nicu-da
6126fc2230
Remove append only prefix from conformance tests [KVL-1152] (#11424)
CHANGELOG_BEGIN
[api-test-tool] - remove AppendOnly prefix from test suites as the append-only schema is the only one left
CHANGELOG_END
2021-10-28 02:24:42 -07:00
Gerolf Seitz
b412f7191d
Fix MonotonicRecordTimeIT for pruned participants (#11423)
Reading explicitly from ledger.begin causes the test to fail on a pruned
participant. The test framework automatically selects a ledger offset
observed at the start of the test anyway, so there's no need to
explicitly put ledger.begin as the starting offset for the checkpoints.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-27 11:47:29 +02:00
Hubert Slojewski
8212c0b2a5
Make submission ID optional [KVL-1107] (#11011)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 14:39:56 +00:00
tudor-da
03cfd1237c
Configurable assertions in Ledger API test tool by feature descriptors (#11328)
* ApiVersionService propagates self-service error codes flag.
* ParticipantTestContext is enriched with feature descriptors
* ContractIdIT adapted with assertions for self-service error codes

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:13:44 +02:00
nicu-da
b1a6b11d47
ledger-api-test-tool: Add static time awareness [KVL-1156] (#11266)
* ledger-api-test-tool: Add command line argument --static-time which makes the test tool aware that the ledger is running in static time

For command deduplication conformance tests this allows us to forward the time so that the tests can run in static time mode as well

CHANGELOG_BEGIN
CHANGELOG_END

* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/Cli.scala

* Do not run conformance test that cannot run in static time

* Remove hidden from static time flag for ledger api test tool

* ledger-api-test-tool: Clarify parts of the deduplication tests.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-10-18 14:16:25 +00:00
fabiotudone-da
d4cb1f945e
KV ledgers: disable participant side command de-duplication [KVL-1083] (#11095)
* Extract common code for command dedup conformance tests

CHANGELOG_BEGIN

CHANGELOG_END

* Revert changes for parallel tests

* Fix compile warnings

* Disable participant deduplication for KV

CHANGELOG_BEGIN
CHANGELOG_END

* Always call submitRequestAndAssertDeduplication with parties

* Fix party in CommandDeduplicationBase

* Don't test KV-based command deduplication with static time as it's broken

* Don't disable participant-side command deduplication for sandbox-next on static time

* Revert "Don't test KV-based command deduplication with static time as it's broken"

This reverts commit 3dac41aa

* Add missing dependency

* Don't cross-test this ledger API test tool with existing versions

* Add the current PR to the list of PRs that resulted in exclusions

* Apply @garyverhaegen-da's suggestion for compatibility exclusions

Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
2021-10-05 13:31:53 +02:00
nicu-da
268b2d3da0
Command deduplication - test case that uses CommandService and CommandSubmissionService [KVL-1106] (#11098)
Add test case for command deduplication which uses both the CommandService and the CommandSubmissionService

CHANGELOG_BEGIN

CHANGELOG_END

* Run mixed CommandService/CommandSubmissionService test for all the possible combinations of calls
2021-10-04 09:42:07 -07:00
nicu-da
5d9ec65cc6
Add test case for parallel command deduplication using mixed clients [KVL-1090] (#11093)
* Add test case for parallel command deduplication using mixed submission through the CommandService and the CommandSubmissionService

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-30 12:10:14 +00:00
nicu-da
517e866df5
Extract common code for command dedup conformance tests [KVL-1090] (#11092)
* Extract common code for command dedup conformance tests

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-30 11:20:46 +00:00
nicu-da
5e424f8d1c
Command dedup conformance suites readme (#11051)
* Add readme which explains the differences between different command deduplication suites

CHANGELOG_BEGIN
CHANGELOG_END

* Update readme
2021-09-29 19:43:17 -07:00
fabiotudone-da
bf801a6e1b
Fix SimpleDeduplicationBasic when participant deduplication is disabled (#11083)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-29 16:58:01 +00:00
nicu-da
7dd9c2d3f0
Remove expectations for internal failures from parallel command dedup tests (#11061)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-29 05:07:01 -07:00
pbatko-da
80e217e11f
[DPP-622] Add conformance tests that verifies TLSv1.0 and TLSv1 are disabled. (#10983)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-27 18:00:52 +02:00
Robert Autenrieth
ced4a27240
Include concurrency info in output (#10970)
changelog_begin
changelog_end
2021-09-22 21:11:26 +02:00
Robert Autenrieth
a0b6800a93
Limit maximum number of concurrent tests (#10969)
* Limit maximum number of concurrent tests

changelog_begin
changelog_end

* Fix CLI description

* Simplify expression

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Remove argument that is not needed anymore

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2021-09-22 21:09:49 +02:00
pbatko-da
e299103450
[Short] Small test tool's CLI parser refactor (#10923) 2021-09-22 11:37:50 +02:00
pbatko-da
855ecdf1a7
[DPP-572] Add ledger API test case for verifying conformance to --min-tls-version flag. (#10898)
Add CLI flag to select minimum enabled TLS version for participant server

CHANGELOG_BEGIN
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
CHANGELOG_END
2021-09-20 12:17:00 +02:00
mziolekda
04d8f753b2
Clean up sandbox-on-x conformance tests. (#10766)
* Clean up sandbox-on-x conformance tests.

CHANGELOG_BEGIN
CHANGELOG_END

* fix a flaky test in config management
2021-09-20 07:52:46 +00:00
nicu-da
b8e21d8f48
Fix takeFilter for the test StreamConsumer (#10918)
* Reorder the observers for StreamConsumer to first filter and then take just one element, not the other way around

CHANGELOG_BEGIN
CHANGELOG_END

* Refactor the test to get better stacktraces during failures

* Remove unused import
2021-09-17 16:16:04 +02:00
nicu-da
f4adee91ca
Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
* Add conformance test for command deduplication using the CommandService

CHANGELOG_BEGIN

CHANGELOG_END
2021-09-16 11:16:39 +00:00
Gerolf Seitz
8a391189a6
Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
* Rename Completion.deduplication_time to deduplication_duration

CHANGELOG_BEGIN
CHANGELOG_END

* Breaking protobuf change: regenerate `buf` image

Breaking-Proto: true

* Re-compute sha256s of migrations

* Fix expected error message

Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
2021-09-16 10:41:45 +00:00
nicu-da
b4328b3dc3
ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication

CHANGELOG_BEGIN

CHANGELOG_END

* Fix broken test and use consistency for tests

* ledger-api-test-tool - Add conformance test for parallel command deduplication

CHANGELOG_BEGIN
CHANGELOG_END

* Add import for 2.12 compat

* Add silencer plugin

* Split parallel command deduplication scenario into it's own test suite

* Add the parallel command deduplication test to append only ledgers

* Run parallel command deduplication tests for append only ledgers

* Apply suggestions from code review

Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>

* Code review renames

* Add compat import

* Run the test concurrently
2021-09-15 12:15:13 +00:00
tudor-da
9a1a1015f2
Increase timeout for heavy tests in ParticipantPruningIT (#10894)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-15 08:52:33 +00:00