* use traverseFM for storeSync
CHANGELOG_BEGIN
CHANGELOG_END
* split the transaction seq trial from the selection of random parameters
* sample focused trial
* temporarily enable focused 48% onlyWildcardParties H2 test in CI
- partially reverts b4244036f6 from #7482
* try different PowerShell syntax, reduce trial count to 250
* remove temporary tests
* metrics: Support tagged Futures when timing.
* ledger-on-sql: Use tagged execution contexts in `Database`.
We have to deal with multiple execution contexts in `Database`. This
makes it possible to use them implicitly, which is much cleaner.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-sql: Simplify `Database` a little.
* ledger-on-sql: Make the connection pool implicit.
* ledger-on-sql: Move the execution context into the connection pool.
* ledger-on-sql: Make connection pools more implicit.
* ledger-on-sql: Use the `sc` prefix for `scala.concurrent`.
* ledger-on-sql: Remove an unnecessary import.
250ms is a bit low for CI when our database might be overloaded.
5 seconds seems like a decent balance between a quick response and
being sympathetic to slow/overloaded machines.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Document the deterministic nature of commit strategies.
* kvutils: Run the commit strategies in as parallel a fashion as possible.
Trees are faster than lists.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Move `serializeStateKey` into its own class.
* add parameter information to "fall back to limit-based query with consistent results" test
* run only one test in CI, and run it a lot more
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* considering a grouped reporter
* never mind that
* clean up the error report
* link to #7521
* remove harder testing
* concurrent: Tag DirectExecutionContext.
1. Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
stating that it works for any tagged `Future`.
2. Move `DirectExecutionContext` to the _libs-scala/concurrent_
library, as it requires it and it's tiny.
CHANGELOG_BEGIN
CHANGELOG_END
* concurrent: Fix the privacy of `DirectExecutionContextInternal`.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* kvutils: Always commit serially.
Committing in parallel will cause the set of writes to be emitted in a
random order.
CHANGELOG_BEGIN
- [Integration Kit] In kvutils, the ``BatchedSubmissionValidator`` no
longer has a parameter for commit parallelism. Commits are now always
written serially to preserve order.
CHANGELOG_END
* kvutils: Sort the output state before writing.
We previously wrote the output state in a random order, leading to
writes that could not be easily compared for integrity. Sorting them
allows us to validate not just the values, but the order.
This means that the exporter no longer sorts the write set, and the
integrity checker does not sort before checking.
CHANGELOG_BEGIN
- [Integration Kit] In kvutils, state is now sorted before committing.
This allows us to provide stronger guarantees with regards to the
serialized write sets.
If you have implemented your own ``CommitStrategy``, you should also
ensure the output state is sorted before committing.
CHANGELOG_END
* ledger-on-sql: Implement SQL typeclasses as values, not functions.
Let's pretend performance is important here.
* kvutils: Allow Iterables in batching ledger state operations.
A `Map[Key, Value]` is an `Iterable[(Key, Value)]`, but not a `Seq`.
This allows us to pass a `Map` straight from the committer.
* kvutils: Construct a `SortedMap` in the commit strategy.
This should be more performant than generating and then sorting a
`Vector`.
* kvutils: Sort state updates when committing after pre-execution.
* participant-integration-api: Inject health checks into the API server.
CHANGELOG_BEGIN
- [Integration Kit] The ``StandaloneApiServer`` now takes a
``healthChecks`` parameter, which should identify the health checks to
be exposed over the gRPC Health Checking Protocol. This will
typically look something like::
healthChecks = new HealthChecks("read" -> readService, "write" -> writeService)
Integrators may also wish to expose the health of more components.
All components wishing to report their health must implement the
``ReportsHealth`` trait.
CHANGELOG_END
* sandbox + kvutils: Add the "read" component back to the health checks.
* remove failedTransactions field from ScenarioLedger.RichTransaction
changelog_begin
changelog_end
* remove Blinding.checkAuthorizationAndBlind; fixup callers to use Blinding.blind when blindningInfo is required
* rename/relocate: ScenarioLedger.CommitError.FailedAuthorizations --> SError.DamlEFailedAuthorization
* fix types to demonstate that at most one FailedAuthorization is detected/reported
* address small review comments
This PR implements a part of the proposal from #7093.
Here packages are validated in the participant node before to be sent to the ledger.
CHANGELOG_BEGIN
- [Ledger-API] participant node validate Dar before uploading to the ledger.
This may increase upload time significantly.
CHANGELOG_END
changelog_begin
changelog_end
Adapt test for small error message change. An improvement! Previously an internal message was shown. Make test flexible enough to pass for old & new message.
Adapt expected output file. Only change is the contract-ids for active-contracts when the test ends.
fix small test bugs in testcase where authorization is wrong
temp disabled 3 tests; needs investigation; see TODO markers
temp adapt 1 test for change in error message
temp disable 1 test. needs invesigation
undo accidentally commited change to .bazelrc
add copyright header to new file
remove testcase (badActorCheck2) which is no longer expected behaviour, when authorization occurs during execution
address comments: be mre private & other tiny changes
appease scala formatter
improve expected error message in KeyNotVisibleStakeholders testcase
fix authorization issues and re-enable the 3 failing tests in EngineTest which now pass
fix auth issue and re-enable ledger-test-tool test: WronglyTypedContractIdIT
fix compatibility
re-enable test in KVUtilsTransactionSpec.scala
I’d like to use this in a test fixture so this PR splits up the main
and exposes an API that gives you back the resource without having to
import Main.
changelog_begin
changelog_end
The migration script from #7344 ensures that there are no more values
in Sandbox Classic DB encoded with Value version < 6. Hence, we do
not need anymore a special EngineConfig to maintain backward
compatibility of Sandbox classic DB.
CHANGELOG_BEGIN
CHANGELOG_END
Before, we could in-place update the value of a var gauge. Now, we can not only set such a value but in-place update it atomically.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-test-tool: Wait for futures when constructing LedgerSession.
* ledger-api-test-tool: Wait for futures on `new ParticipantSession`.
* ledger-api-test-tool: If we create two sessions, close one immediately.
* ledger-api-test-tool: Only create sessions once.
CHANGELOG_BEGIN
- [Ledger API Test Tool] Fix a race condition in which multiple
connections were created to a single participant, and only one was
shut down properly. This error was likely benign but may cause
spurious errors to show up when the test tool finishes and shuts down.
CHANGELOG_END
* ledger-api-test-tool: Let ParticipantSessionManager manage connections.
* ledger-api-test-tool: Shrink the configuration surface area.
* ledger-api-test-tool: Use `Future.traverse` to simplify LedgerSession.
* ledger-api-test-tool: Fix the ParticipantSessionManager logger.
* ledger-api-test-tool: Address issues flagged by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-api-test-tool: Make `SessionParts` a real class.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Extend wait time for dedupe tests to 2x dedupe window
* default clock tick interval of 10s, make it optionally configurable
CHANGELOG_BEGIN
[ledger-api-test-tool] Include new attribute for configurable clock tick
interval
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* address review comments
* Address code review - use Duration
* Add documentation on --ledger-clock-granularity option
* Update documentation
* address review comments
* address warning: implicit numeric widening
* scalafmt
* ProtobufDuration alias rather than ProtoDuration for consistency with namespacing
* underline in docs
This PR uses the new data structure introduced in #7220.
Additionnally this fix `Value Equal instance` which was considering
<a: x, b: y> different from <b:y, a:x>.
CHANGELOG_BEGIN
CHANGELOG_END
* Canton conformance test debug logging and reduced test parallelism
To reign in flakiness of a handful of TransactionServiceIT tests
that had run into timeouts during synchronize
Closes#7316
changelog_begin
changelog_end
* Review feedback from Stefano
* mark canton-conformance-test as flaky
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz's review
* fix
* fix
* Disable a handful of TransactionServiceIT changes that flake
rather than making the entire canton suite as flaky
Related to #7316
changelog_begin
changelog_end
* formatting
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
* uniquely name tests to easily facilitate running them one by one for troubleshooting
CHANGELOG_BEGIN
CHANGELOG_END
Signed-off-by: Brian Healey <brian.healey@digitalasset.com>
* ensure a few further tests do not include a portion of another tests shortIdentifier
* no overlap for CSsubmitAndWaitForTransactionTree
* kvutils/tools: Split the integrity checkers by package.
The two versions of the integrity checker are intertwined when they
really don't rely on each other at all. This splits them into two
separate packages to make the distinction clear.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils/tools: Split the integrity checkers by directory.
This splits the integrity checkers further into separate directories, to
make it clear they do not interact at all.
* kvutils/tools: Rename "integrity-check" to "integrity-check-v1".
* ledger-on-memory: Recommend ledger exports (v3), not ledger dumps (v1).
* kvutils/tools: Split the benchmarks from integrity-check-v2.
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.
CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
to ``maxInboundMetadataSize``, to match the builder Netty channel
builder. It was incorrectly named ``maxInboundMessageSize``, which is
a different channel property that configures the maximum message size,
not the header size.
CHANGELOG_END
* ledger-api-client: Introduce a `maxInboundMessageSize` config property.
We use this a lot; easier if it's in the configuration.
CHANGELOG_BEGIN
- [Scala Bindings] Replace the
``LedgerClientConfiguration.maxInboundMessageSize`` property with a
new one that represents the maximum size of the response body.
CHANGELOG_END
* kvutils: Implement a basic protocol buffer-based export.
* kvutils: Add a header to the protobuf-based export.
So if we decide to go for a v4, it's easier to figure this stuff out.
* kvutils: Use our own header format for export.
The idea is to be somewhat forward-compatible, so relying on protobuf
to always be the format of the future seems a little odd. We may switch.
* kvutils: Construct the importer up-front during integrity checks.
* kvutils: Move the various importers and exporters to versioned packages.
* kvutils: Move the export version to the package object.
* kvutils: Create helpers to construct importers and exporters from paths.
* kvutils: Process either v2 or v3 exports with the integrity checker.
* kvutils: Sort the write set during export.
* kvutils: Switch to v3 of the export.
The integrity checker will work with either version.
CHANGELOG_BEGIN
- [Integration Kit] kvutils-based ledgers will now export the ledger
using a new serialization format based on Protocol Buffers, which we
are referring to as "v3". Existing "v2" exports can still be read and
verified using the integrity checker, which can read both old and new
versions.
CHANGELOG_END
* kvutils: Aggregate export data in a sorted map.
We need to sort it anyway; might as well do it on input.
* kvutils: Improve a test string.
* kvutils: Use the v3 importer in the replay test.
* kvutils: Remove v2 of the export format.
We're pretty sure no one is using it.
The executor may no longer be available. Fortunately we can avoid using
resources entirely in this case, and just acquire the channel the
old-fashioned way.
CHANGELOG_BEGIN
- [Scala Bindings] Fixed a bug in the ledger client which meant that the
client was not closed properly on application shutdown. You may have
seen some ``RejectedExecutionException`` errors in your logs if this
has affected you. We encourage all users of the `LedgerClient` to call
the ``close`` method explicitly to ensure it is closed at an
appropriate time in your application lifecycle.
CHANGELOG_END
* ledger-on-(memory + sql): Include logback as a test dependency.
Otherwise logs don't show up when we get failures, which makes debugging
a lot harder.
* kvutils: On unit test timeout, show a useful error.
* kvutils: Increase the streaming updates timeout in integration tests.
This is timing out every now and again on CI, and I can reproduce it
easily by running the tests in parallel.
CHANGELOG_BEGIN
CHANGELOG_END
* Refactor EnrichedTransaction
This PR prepares for work to interleave _authorization_ with _execution_, which has been an open issue for a long time: #132.
- Split computation of `BlindingInfo` and `FailedAuthorizations` into separate modules.
- Fixup callers: `Blinding.scala` and `ScenarioLedger.scala`, to do:
```
val failedAuthorizations =
AuthorizingTransaction.checkAuthFailures(authorization, tx)
val blindingInfo =
BlindingTransaction.calculateBlindingInfo(tx)
```
- Simplify code to remove `DontAuthorize` and `Authorization` types, which were only needed to restrict the combined _enrichment_ code from doing the authorization check when `blind`ing only is required. Now `Blinding.blind` calls just `calculateBlindingInfo`.
* address comments from Stefano
* kvutils: Factor out a base class for ledger exporter implementations.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Use piped streams in LedgerDataExporterSpecBase.
* kvutils: Rename "FileBased…" to "SerializationBasedLedgerDataExporter".
* kvutils: Wrap Deserialization in a class for polymorphism purposes.
* kvutils: Pull out submission info variables in the ledger export test.
* kvutils: Inline (De)Serialization into the exporter.
* kvutils: Replace usage of Deserialization with LedgerDataImporter.
* kvutils: Don't synchronize on making an InMemorySubmissionAggregator.
* kvutils: Pass a SubmissionInfo to LedgerDataExporter.
* kvutils: Reflow a long string to make it readable.
It turns out that if you give the CSV reporter a non-existent directory,
it crashes. I did not expect this.
This constructs the directory so you don't have to worry about that.
CHANGELOG_BEGIN
CHANGELOG_END
This is the same technique as `DerivativeGauge` from the metrics
library, but with less code, because Scala is prettier than Java.
CHANGELOG_BEGIN
CHANGELOG_END
