* [DPP-417][DPP-613] Adopt self-service error codes ApiTransactionService
CHANGELOG_BEGIN
CHANGELOG_END
* TV review
* TV review
* Uses ErrorFactories
* Error loggers use enriched logging contexts
* Deleted ApiTransactionServiceSpec
* Add TODO wrt to CommandValidation.invalidArgument
* Adapt transactionNotFound error assertions in conformance tests
* ErrorFactories.invalidArgumentWasNotFound for handling invalid event ids
* Revert improvement to TransactionNotFound cause message since
Canton still outputs the old message and assertions fail.
Co-authored-by: Tudor Voicu <tudor.voicu@digitalasset.com>
* [Self-service error codes] Implement V2 in Authorizer
CHANGELOG_BEGIN
CHANGELOG_END
* Added unit test for authorize (non-streamed)
* Fix after rebase
* Do not expose the error codes switching mechanism to the Java bindings
* Adjust InternalAuthorizationError to be SystemInternalAssumptionViolated
* Parameter names in test
* Testing AuthorizationInterceptor with regard to returned error codes
* Do not use default error code version switchers at instance creation
* Addressed Pawel's review comments
* Using ErrorFactories for error dispatching
* Pass loggingContext to Authorizer where available
* Generic internal authorization error
Adding `Daml Documentation >> Building applications >> "Self-Service Error Codes (Experimental)` section in the HTML documentation.
The section is populated automatically with error code information retrieved from the classpath.
The process of generating documentation for error codes looks like this:
- first we find error codes information from the classpath,
- then we save it to a json file,
- then the json file is made available to a custom Sphinx extension,
- then the custom Sphinx extension generates documentation wherever we put a new custom Sphinx directive.
Try it out with
`./docs/scripts/live-preview.sh`
or
`./docs/scripts/preview.sh`
CHANGELOG_BEGIN
CHANGELOG_END
* Support contextual logging in KeyValueConsumption
CHANGELOG_BEGIN
CHANGELOG_END
* Fix formatting and imports
* Propagate the contextual logging per-readUpdates call
* No need for at-construction logging context in LogAppendingReadServiceFactory
* No need for logging context in CommitStrategySupport.newReadServiceFactory
* Fix JdbcIndexerSpec
There is no longer a non-versioned offset, so this is kind of
meaningless.
I also deleted `VersionedOffsetMutator`, which should not have made it
in.
CHANGELOG_BEGIN
CHANGELOG_END
* Propagate error version switch to KeyValueConsumption
CHANGELOG_BEGIN
CHANGELOG_END
* Introduce v2 (self-service) KV error codes behind the CLI switch
* Test v2 codes and fix them
* Keep newline separating methods in KeyValueConsumption
* TransactionRejections: don't wrap updates in `Some`
* Factor errorVersionsTable
* Reorder imports
* Split "convert rejection to proto models and back to expected grpc code"
* Remove unneeded Option returned by decodeTransactionRejectionEntry
* Formatting fix
* Fix 7537e93d
* kvutils: Remove `VersionedOffsetBuilder.apply`.
Doesn't do anything.
* ledger-on-memory: Use `VersionedOffsetBuilder`.
* indexer-benchmark: Use `VersionedOffsetBuilder`.
* ledger-on-sql: Use `VersionedOffsetBuilder`.
* kvutils: Use `VersionedOffsetBuilder` in tests.
* kvutils: Create a case class for `VersionedOffsetBuilder#split`.
* kvutils: Delete unused methods from the offset builders.
* kvutils: Use `Bytes#startWith` to check the offset version.
* kvutils: Move offset splitting into `VersionedOffset`.
* kvutils: Extract out versioned offset generators.
* kvutils: Replace `OffsetBuilder` with `VersionedOffsetMutator`.
This takes care of the last usages of `OffsetBuilder`, which were to
modify the lowest component of the offset.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Randomize the offset version in testing.
To make sure we don't use a hard-coded offset version anywhere.
* kvutils: `Random.between` is not available in Scala 2.12.
* kvutils: Move offset mutation methods to `VersionedOffset`.
* kvutils: Move the versioned offset construction into `VersionedOffset`.
The `VersionedOffsetBuilder` is still useful as it remembers the
version, meaning we only need to specify it once.
* Moved ErrorCodesVersionSwitcher to //ledger/error
CHANGELOG_BEGIN
CHANGELOG_END
* Rename ErrorCodeLoggingContext to ContextualizedErrorLogger
* Refactored ErrorFactories
* All error factories use ContextualizedErrorLogger for being able to dispatch self-service error codes.
* The ContextualizedErrorLogger is passed down from the dispatching Ledger API services.
* ErrorFactoriesSpec asserts both legacy (V1) and self-service error codes (V2).
* Adapted ApiSubmissionService
* Addressed Marcin's review comments
* ledger-api-test-tool: Add command line argument --static-time which makes the test tool aware that the ledger is running in static time
For command deduplication conformance tests this allows us to forward the time so that the tests can run in static time mode as well
CHANGELOG_BEGIN
CHANGELOG_END
* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/Cli.scala
* Do not run conformance test that cannot run in static time
* Remove hidden from static time flag for ledger api test tool
* ledger-api-test-tool: Clarify parts of the deduplication tests.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
On CI we’ve seen
//ledger/ledger-on-memory:ledger-on-memory-tests_test_suite_src_test_suite_scala_com_daml_ledger_on_memory_InMemoryLedgerReaderWriterIntegrationSpec.scala
run above the 60s timeout.
Even locally under load, it gets easily between 30s and 60s so CI
timing out is fairly reasonable.
changelog_begin
changelog_end
* [Self-service error codes] Refactored for Canton integration
* Introduces ErrorCodeLoggingContext which is used to abstract away the logging tech stack
CHANGELOG_BEGIN
CHANGELOG_END
* Moved all error definitions in //ledger/error
* Formatting
* Docs and remove of implicit conversion to DamlErrorCodeLoggingContext
* Fix compilation issues
* Fix tests
* Addressed Pawel's review comments
* Addressed Robert's review comments
* Use append-only schema in tests
changelog_begin
changelog_end
* Fix Oracle build
* Update ledger/ledger-on-memory/BUILD.bazel
Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
* Run more conformance tests on append-only
* Run pruning tests in split-participant mode
* Disable multi-participant test in split participant
* Add missing ContractIdIT test
Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
* Removes flag and related configuration
* Removes unnecessary keepAlive approach from indexer
CHANGELOG_BEGIN
Participant HA feature added: Indexer component is guarded by database locks, and executes in a critical section (prevents overlapping indexer activity). Supported by Oracle and Postgresql database backends, and not supported by H2.
CHANGELOG_END
* kvutils: Tidy up LogAppendingReadServiceFactorySpec.
* kvutils: Add a `SubmissionInfo` parameter to the replaying read service.
Required for other KV ledger implementations.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Rename some constants in LogAppendingReadServiceFactorySpec.
* Rename Completion.deduplication_time to deduplication_duration
CHANGELOG_BEGIN
CHANGELOG_END
* Breaking protobuf change: regenerate `buf` image
Breaking-Proto: true
* Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result
The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.
CHANGELOG_BEGIN
CHANGELOG_END
* Do not call duplicate command exception twice
* Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness
CHANGELOG_BEGIN
CHANGELOG_END
* Make schema migration retries and backoff configurable
* Review feedback - use RetryStrategy.constant instead
* Remove unused tailrec import
* Simplifications and runF by Marton
* Rename config options from retry to attempt and default to 30 attempts
* Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing
* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Fix broken test and use consistency for tests
* ledger-api-test-tool - Add conformance test for parallel command deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Add import for 2.12 compat
* Add silencer plugin
* Split parallel command deduplication scenario into it's own test suite
* Add the parallel command deduplication test to append only ledgers
* Run parallel command deduplication tests for append only ledgers
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Code review renames
* Add compat import
* Run the test concurrently
* Further rename completions' `deduplication_time` to `*_duration` in participant's codebase
CHANGELOG_BEGIN
CHANGELOG_END
* Re-compute sha256s of migrations
* Fix expected error message
* Improve consistency of "deduplication time" Vs. "deduplication duration" references
* Revert Oracle schema whitespace change
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
Co-authored-by: nicu-da <nicu.reut@digitalasset.com>
Co-authored-by: Oliver Seeliger <46452754+oliverse-da@users.noreply.github.com>
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
* Extract common code for command dedup conformance tests
CHANGELOG_BEGIN
CHANGELOG_END
* Revert changes for parallel tests
* Fix compile warnings
* Disable participant deduplication for KV
CHANGELOG_BEGIN
CHANGELOG_END
* Always call submitRequestAndAssertDeduplication with parties
* Fix party in CommandDeduplicationBase
* Don't test KV-based command deduplication with static time as it's broken
* Don't disable participant-side command deduplication for sandbox-next on static time
* Revert "Don't test KV-based command deduplication with static time as it's broken"
This reverts commit 3dac41aa
* Add missing dependency
* Don't cross-test this ledger API test tool with existing versions
* Add the current PR to the list of PRs that resulted in exclusions
* Apply @garyverhaegen-da's suggestion for compatibility exclusions
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
