* Create single-party specialized queries to access events
Closes#5620
changelog_begin
changelog_end
* Ensure ordering of events when performing a lookup
- removes unused `is_root` column, closes#5618
- merges `template_package_id` and `template_name` in `template_id`, closes#5619
changelog_begin
[Ledger API] clean up of the participant index, improved performance when requesting transactions by template
changelog_end
Instead, opt-in explicitly in _ledger-on-memory_ and _ledger-on-sql_.
Wrapping the operations can confuse other users of `LedgerStateAccess`.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-common: Add a metric for command submission validation.
CHANGELOG_BEGIN
- [Ledger API Server] Add a metric for command validation upon
submission, ``daml.commands.validation``.
CHANGELOG_END
* ledger-api-common: Move the submissions metric from sandbox.
* sandbox: Add a test for MetricsInterceptor for non-streaming endpoints.
* sandbox: Simplify MetricsInterceptor by overriding `ServerCall`.
* sandbox: Time all submissions, not just through the submission service.
* sandbox: Don't bother timing transaction submissions twice.
They're already timed by TimedWriteService.
* sandbox: Move command submission metrics to "daml.commands".
CHANGELOG_BEGIN
- [Ledger API Server] A number of command submission metrics have moved,
and we have added a couple.
- ``daml.commands.submissions`` is a new timer that measures all
submissions.
- ``daml.commands.valid_submissions`` is a new meter that counts
valid (unique, interpretable) submissions.
- ``daml.lapi.command_submission_service.failed_command_interpretations``
has been moved to ``daml.commands.failed_command_interpretations``.
- ``daml.lapi.command_submission_service.deduplicated_commands`` has
been moved to ``daml.commands.deduplicated_commands``.
- ``daml.lapi.command_submission_service.delayed_submissions`` has
been moved to ``daml.commands.delayed_submissions``.
- ``daml.lapi.command_submission_service.submitted_transactions`` has
been moved to ``daml.services.write.submit_transaction``.
- All ``daml.execution.*`` metrics have moved to
``daml.commands.execution``.
CHANGELOG_END
* metrics: Update the dashboard to include the new command metrics.
* Add time provider type to ledger API server
Static time sometimes needs special treatment
* Delay submissions of transactions with future time
Fixes#5480.
CHANGELOG_BEGIN
- [Sandbox] The sandbox now properly delays command submissions using
minLedgerTimeAbs or minLedgerTimeRel. See
`issue #5480 <https://github.com/digital-asset/daml/issues/5480>`_.
CHANGELOG_END
* ledger-on-memory: Add metrics to the operations.
* ledger-on-memory: Speed up reads by using views.
* kvutils: Time event processing.
* ledger-on-(memory|sql): Make classes private and final where possible.
* kvutils: Factor out a TimedLedgerStateOperations class.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
operations.
CHANGELOG_END
* ledger-on-sql: Record log read metrics.
* ledger-on-sql: Record database transaction timing metrics.
CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END
* ledger-on-sql: Time queries.
* metrics: Add graphs for read events and DAML-on-SQL.
* ledger-on-memory: Simplify the tuple swap in reading the log.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* sandbox: Construct PartyConfiguration later.
* sandbox: Provide an option to disable implicit party allocation.
CHANGELOG_BEGIN
- [Sandbox] You can now disable implicit party allocation by passing the
flag, ``--implicit-party-allocation=false``. This makes it easier to
test as you would against another ledger which does not support this
feature.
CHANGELOG_END
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
We started serving the LedgerAPI from the new schema in #5104, #5305, #5346,
and #5397.
We must therefore also add a migration that moves the data from
ledger_entries to the new tables.
To keep the migration relatively stable with respect to refactoring the
code, I copied a bunch of files to
`db.migration.postgres.v25_backfill_participant_events`.
As a bonus, I added a line to logback.xml so that we get the usual
sandbox output on startup.
CHANGELOG_BEGIN
CHANGELOG_END
* participant_contracts write path
Writes contracts alongside events for interpretation and validation.
changelog_begin
changelog_end
* Fix self-referencing foreign key
* Address https://github.com/digital-asset/daml/pull/5522#discussion_r406289145
* Compute divulgence only if necessary
* Do not attempt at persisting witnesses for transient contracts
* Drop new tables when resetting
* sandbox: Time command execution.
* sandbox: Tiny cleanups in StoreBackedCommandExecutor.
* sandbox: Add metrics for fetching within execution.
CHANGELOG_BEGIN
- [Ledger API Server] Add metrics under "daml.execution" to measure
command execution time, and fetching within command execution.
CHANGELOG_END
* sandbox: Revert the ApiSubmissionService metric prefix.
It can be private again.
* sandbox: Record a mark on a meter on command execution retries.
- removes unnecessary reference to offsets
- pushes event identifier translation out to enable code re-use
- event identifier translation is inline in other iterations, no significant perf degradation
- this will allow to share code between event and contract witness table objects
changelog_begin
changelog_end
* sandbox: Remove `ErrorCause.Sequencer`.
It doesn't seem to be constructed anywhere.
Feel free to close this if it's used outside the repository.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Don't deconstruct when we can check types in a pattern match.
* sandbox: Move command execution into its own package.
* sandbox: Make LedgerTimeHelper into a CommandExecutor implementation.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Rename CommandExecutorImpl to StoreBackedCommandExecutor.
And pass it objects, not functions.
* sandbox: Reorder result cases in StoreBackedCommandExecutor.
* sandbox: Inject the LedgerTimeAwareCommandExecutor.
* sandbox: Pull out the default time provider type into a constant.
* sandbox: Name ResetService tests consistently.
* sandbox: Reset the time service when resetting in Static Time mode.
The test, unfortunately, is ``@Ignore`d due to flakiness in CI, so won't
actually be run. However, I _hope_ we're going to remove that
annotation eventually, and it allowed me to test-drive the fix on my
machine, so it's still helpful.
CHANGELOG_BEGIN
- [Sandbox] Fix a regression in the ResetService which did not reset the
TimeService in static time mode.
CHANGELOG_END
Turns out not every participant can support seeding (yet).
CHANGELOG_BEGIN
- [Ledger API Server] Re-introduce an option to disable seeding. This
does not affect Sandbox.
CHANGELOG_END
* Set the `Bearer ` prefix in bindings.
* Make the `Bearer ` prefix in the authorization header mandatory.
* Bearer prefix can be removed from the token file.
CHANGELOG_BEGIN
[Extractor]: The ``Bearer `` prefix can be removed from the token file.
It is added automatically.
[Navigator]: The ``Bearer `` prefix can be removed from the token file.
It is added automatically.
[DAML Script] The ``Bearer `` prefix can be removed from the token file. It
is added automatically.
[DAML Repl] The ``Bearer `` prefix can be removed from the token file. It is
added automatically.
[Scala Bindings] The ``Bearer `` prefix can be removed from the token. It is
added automatically.
[Java Bindings] The ``Bearer `` prefix can be removed from the token. It is
added automatically.
[DAML Integration Kit] ``AuthService`` implementations MUST read the
``Authorization`` header and the value of the header MUST start with
``Bearer ``.
CHANGELOG_END
* Produce performance tests for all envelopes
CHANGELOG_BEGIN
[TestTool] Provide performance tests for all performance envelopes
CHANGELOG_END
* Follow Scala's recommended pattern for enum-like hierarchies
There have been reports of sporadic occurrences of
`java.lang.RuntimeException: Lob not found: 49/-2`
when running the participant server with H2.
It seems like using `setBinaryStream` triggers H2 to go through the BLOB
machinery. Since this issue is not easily reproducible, it seems like an
altogether better solution to switch to using `setBytes`. Offsets aren't
that large anyway, so going directly for byte array should be fine.
Better than a broken query anway.
CHANGELOG_BEGIN
CHANGELOG_END
* Add failing test
The test can produce false negatives,
but locally it fails 10 out of 10 times.
* Stop deduplicating commands after rejections
Fixes#5338.
CHANGELOG_BEGIN
CHANGELOG_END
* Only deduplicate successful transactions
* Use pass instead of pure unit
* Handle exceptions
* kvutils: Simplify calculating the weight of a Caffeine cache.
And remove an errant `println` that slipped through the cracks.
Thank you to @ben-manes for the tip!
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Make classes final and defs into vals.
Co-Authored-By: Stefano Baghino <stefano.baghino@digitalasset.com>
Co-authored-by: Stefano Baghino <stefano.baghino@digitalasset.com>
We do this for Sandbox Next™; we can do it for Sandbox Classic™ too. It
doesn't seem to have much of a noticeable impact on the test run time,
and reduces the number of exclusive tests, which is helpful.
CHANGELOG_BEGIN
CHANGELOG_END
* Add legacy proxy gRPC services.
This exposes the services as com.digitalasset as well, to ensure that
applications built with a previous release of the SDK continue to work
with the Ledger API.
Due to how the gRPC reflection service works, this doesn't expose the
com.digitalasset services on the reflection api, and thus grpcurl won't
work with the old services. These scripts need to be updated to refer to
the com.daml services.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Cache state value conversions from bytes.
This seems to have a decent speedup in ledger-on-memory.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Submissions now look up ledger values from a
cache where possible, improving performance when there's contention over
certain resources (e.g. common packages). The cache size currently
defaults to 64 MB.
CHANGELOG_END
* kvutils: Make the SubmissionValidator statue value cache configurable.
* kvutils: Report state value cache metrics.
* kvutils: Add a suffix to a Long literal because WartRemover is unhappy.
Strangely, it doesn't fail on my machine.
* kvutils: Extract caching out into its own file.
* kvutils: Move the `bytesToStateValue` call into `cache.get`.
* kvutils: Move caching to its own package.
* kvutils: Inject the state value cache.
* kvutils: Default to no state value cache.
* kvutils: Accept a state value cache size in megabytes, not bytes.
* kvutils: Move cache building from `Config` to the `caching` package.
* kvutils: Replace Guava's cache with Caffeine.
* kvutils: Simplify caching configuration.
* sandbox: Enable state value caching by default.
CHANGELOG_BEGIN
- [Sandbox] State values deserialization is now cached, with a fixed
cache size of 128MB.
CHANGELOG_END
* Changelog commit.
CHANGELOG_BEGIN
- [Ledger Integration Kit] The state value cache is now opt-in, with a
default of no cache at all.
CHANGELOG_END
* Draft of PingPong throughput and latency benchmarks for on-mem and on-sql ledgers
* Augment `ParticipantTestContext` and remove `LedgerApiServer` hack
* Separate performance tests into distinct category with concurrency 1
* 🎨
* Package performance tests in separate DAR
* Have performance tests excluded by default and run exclusively if passed
* Fix rebase
* Simplify `BenchmarkReporter`
* Make `concurrencyOverride` into an `Option`
* Clarify command line usage, prevent regular and perf. tests together
* Fix preventing regular and perf. tests together
* Split `PingPong`, `PingPongExplode` and `Cycle` benchmarks' model
CHANGELOG_BEGIN
- [TestTool] Add `PingPong` performance envelope test
CHANGELOG_END
* Explicitly name `concurrencyOverride`
* Fix formatting
* Lower bar for CI run of performance envelope tests
* Make benchmark output file configurable
* Improve messages and report config option name
* Use exit status 64 for "bad command line usage" as in BSD
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
* Fix test for groupContiguous
Automatic generation of test values was prone to cause flakiness, removed in favor of a simpler test case.
changelog_begin
changelog_end
* Relax order sensitivity
* Update ledger/sandbox/src/test/suite/scala/com/digitalasset/platform/store/dao/events/GroupContiguousSpec.scala
Co-Authored-By: Remy <remy.haemmerle@daml.com>
* Fix compilation issue
Co-authored-by: Remy <remy.haemmerle@daml.com>
* sandbox: Move the events page size configuration value into config.
* sandbox: Pass `config` directly into JdbcIndexerFactory.
* sandbox: Reorder `eventsPageSize` before `metrics` in parameters.
* sandbox: Move `seeding` into `ApiServerConfig`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Name all parameters of `JdbcLedgerDao.writeOwner`.
Co-Authored-By: stefano.baghino@digitalasset.com
* Implement timed command deduplication in kvutils
This adds a field deduplication_time to DamlCommandDedupValue for
deduplication timeout checking.
* Bump kvutils version to 4
* Fix CommandTracker pulling commandResultIn multiple times
Now that the timeouts are generated out of band, we have 2
"unsynchronized" places that pull on commandResultIn.
Whenever we pull, we need to check that commandResultIn hasn't been
pulled before.
* Add inStaticTimeMode flag to enable command dedup in sandbox-next with static-time
Fixes#4624.
CHANGELOG_BEGIN
[kvutils] KVUtils now respects the command deduplciation time instead of
deduplicating commands forever.
CHANGELOG_END
