* sandbox: Split up LedgerApiServer.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Extract out API service construction from LedgerApiServer.
* sandbox: Pull out a case class for GrpcServerOwner event loop params.
* sandbox: Split ownership of `ApiServices` from the object.
In #5419 we set specific loggers to WARN or ERROR so that we avoid noisy
logging by default. However, if the user specifies the --log-level CLI
param, not only the root logger, but also other loggers set in logback.xml
should log at that level.
CHANGELOG_BEGIN
[Sandbox] Properly respect the --log-level CLI parameter
CHANGELOG_END
The duplicated information was not always in sync.
Fixes#5662
CHANGELOG_BEGIN
- [Sandbox] Fix an issue where the sandbox would sometimes reject transactions
with a "INVALID_ARGUMENT: Disputed" error if either of
``min_ledger_time_rel`` or ``min_ledger_time_abs`` was set in the
submission request.
CHANGELOG_END
* Fix hashing of keys in the new contract store
Use GlobalKey.hash instead of KeyHasher.hashKey
Fixes#5680
changelog_begin
changelog_end
* Fix order of fields in migration
* Fix migration naming
* Rename file according to class
* Fix hash of active contracts, not consumed ones
* Make the contract keys test fail if the transaction is not rejected by the committer
- First, this removes the assert statement in the DAML model, so that the
transaction validation doesn't fail with a transaction abort.
- Second, the gRPC error message should contain Disputed. `InvalidLookup`
should only be used in Sandbox Classic, but due to code sharing, the
validation that triggers this error is also run for Sadbonx.
* Fix the key lookup in kvutils
The current way of transaction validation and key lookup does not work
in the following scenario:
- the transaction coming from the participant has a negative key lookup
- the contract key state is loaded, which points to a contract that has
been created by another transaction since the command interpretation
on the participant
- the contract state for the contractId inside the contract key state
has not been loaded, therefore the visibility and activeness check in
`ProcessTransactionSubmission.lookupKey` returns `None`, even though
there is in fact a contract with such a key.
To mitigate this issue, the `lookupKey` function must not check
activeness or visibility. If the submitter is not allowed to load the
contract via a key lookup, the authorization check should catch that.
Any other situation that is inconsistent should result in a rejection of
the transaction.
A small side effect of this change is that the `lookupContract` function
might try to load a contract that wasn't loaded from the kv state
earlier. This only happens because we first rebuild the entire
transaction before we compare it to the original transaction.
Previously we threw an exception in this case, but this results in a rather
ugly abort of the reinterpretation of the transaction. Therefore the
`lookupContract` function just returns `None`. Eventually the
transaction will be rejected during validation of the nodes, because
the `LookupByKey` nodes will be different.
This would not happen if we reinterpreted the transaction lazily and
compared the nodes as soon as they are produced with the corresponding
node of the original transaction.
CHANGELOG_BEGIN
[kvutils] Fixed the validation of key lookups.
See issue `#5562 <https://github.com/digital-asset/daml/issues/5562>`__.
[Participant Server, Ledger API Test Tool] Use rejection reasons of the participant-state API instead of sandbox specific ones.
CHANGELOG_END
* Validate causal monotonicity of used contract keys
When using LookupByKey, the return contract isn't necessarily fetched or used
any other way. This means that we need an extra pass over all contract keys to
ensure that causal monotonicity is respected. To be able to do this,
there is now a new field `DamlContractKeyState.active_at`, which is set to the
same value `DamlTransactionEntry.ledger_effective_time`.
An additional check in `ProcessTransactionSubmission` uses this field to verify
causal monotonicity.
This could lead to slow rollbacks that could cascade on errors that
pile up and keep precious DB threads busy for longer than needed.
changelog_begin
changelog_end
* Create single-party specialized queries to access events
Closes#5620
changelog_begin
changelog_end
* Ensure ordering of events when performing a lookup
- removes unused `is_root` column, closes#5618
- merges `template_package_id` and `template_name` in `template_id`, closes#5619
changelog_begin
[Ledger API] clean up of the participant index, improved performance when requesting transactions by template
changelog_end
Instead, opt-in explicitly in _ledger-on-memory_ and _ledger-on-sql_.
Wrapping the operations can confuse other users of `LedgerStateAccess`.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-common: Add a metric for command submission validation.
CHANGELOG_BEGIN
- [Ledger API Server] Add a metric for command validation upon
submission, ``daml.commands.validation``.
CHANGELOG_END
* ledger-api-common: Move the submissions metric from sandbox.
* sandbox: Add a test for MetricsInterceptor for non-streaming endpoints.
* sandbox: Simplify MetricsInterceptor by overriding `ServerCall`.
* sandbox: Time all submissions, not just through the submission service.
* sandbox: Don't bother timing transaction submissions twice.
They're already timed by TimedWriteService.
* sandbox: Move command submission metrics to "daml.commands".
CHANGELOG_BEGIN
- [Ledger API Server] A number of command submission metrics have moved,
and we have added a couple.
- ``daml.commands.submissions`` is a new timer that measures all
submissions.
- ``daml.commands.valid_submissions`` is a new meter that counts
valid (unique, interpretable) submissions.
- ``daml.lapi.command_submission_service.failed_command_interpretations``
has been moved to ``daml.commands.failed_command_interpretations``.
- ``daml.lapi.command_submission_service.deduplicated_commands`` has
been moved to ``daml.commands.deduplicated_commands``.
- ``daml.lapi.command_submission_service.delayed_submissions`` has
been moved to ``daml.commands.delayed_submissions``.
- ``daml.lapi.command_submission_service.submitted_transactions`` has
been moved to ``daml.services.write.submit_transaction``.
- All ``daml.execution.*`` metrics have moved to
``daml.commands.execution``.
CHANGELOG_END
* metrics: Update the dashboard to include the new command metrics.
* Add time provider type to ledger API server
Static time sometimes needs special treatment
* Delay submissions of transactions with future time
Fixes#5480.
CHANGELOG_BEGIN
- [Sandbox] The sandbox now properly delays command submissions using
minLedgerTimeAbs or minLedgerTimeRel. See
`issue #5480 <https://github.com/digital-asset/daml/issues/5480>`_.
CHANGELOG_END
* ledger-on-memory: Add metrics to the operations.
* ledger-on-memory: Speed up reads by using views.
* kvutils: Time event processing.
* ledger-on-(memory|sql): Make classes private and final where possible.
* kvutils: Factor out a TimedLedgerStateOperations class.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
operations.
CHANGELOG_END
* ledger-on-sql: Record log read metrics.
* ledger-on-sql: Record database transaction timing metrics.
CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END
* ledger-on-sql: Time queries.
* metrics: Add graphs for read events and DAML-on-SQL.
* ledger-on-memory: Simplify the tuple swap in reading the log.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* sandbox: Construct PartyConfiguration later.
* sandbox: Provide an option to disable implicit party allocation.
CHANGELOG_BEGIN
- [Sandbox] You can now disable implicit party allocation by passing the
flag, ``--implicit-party-allocation=false``. This makes it easier to
test as you would against another ledger which does not support this
feature.
CHANGELOG_END
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
We started serving the LedgerAPI from the new schema in #5104, #5305, #5346,
and #5397.
We must therefore also add a migration that moves the data from
ledger_entries to the new tables.
To keep the migration relatively stable with respect to refactoring the
code, I copied a bunch of files to
`db.migration.postgres.v25_backfill_participant_events`.
As a bonus, I added a line to logback.xml so that we get the usual
sandbox output on startup.
CHANGELOG_BEGIN
CHANGELOG_END
* participant_contracts write path
Writes contracts alongside events for interpretation and validation.
changelog_begin
changelog_end
* Fix self-referencing foreign key
* Address https://github.com/digital-asset/daml/pull/5522#discussion_r406289145
* Compute divulgence only if necessary
* Do not attempt at persisting witnesses for transient contracts
* Drop new tables when resetting
* sandbox: Time command execution.
* sandbox: Tiny cleanups in StoreBackedCommandExecutor.
* sandbox: Add metrics for fetching within execution.
CHANGELOG_BEGIN
- [Ledger API Server] Add metrics under "daml.execution" to measure
command execution time, and fetching within command execution.
CHANGELOG_END
* sandbox: Revert the ApiSubmissionService metric prefix.
It can be private again.
* sandbox: Record a mark on a meter on command execution retries.
- removes unnecessary reference to offsets
- pushes event identifier translation out to enable code re-use
- event identifier translation is inline in other iterations, no significant perf degradation
- this will allow to share code between event and contract witness table objects
changelog_begin
changelog_end
* sandbox: Remove `ErrorCause.Sequencer`.
It doesn't seem to be constructed anywhere.
Feel free to close this if it's used outside the repository.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Don't deconstruct when we can check types in a pattern match.
* sandbox: Move command execution into its own package.
* sandbox: Make LedgerTimeHelper into a CommandExecutor implementation.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Rename CommandExecutorImpl to StoreBackedCommandExecutor.
And pass it objects, not functions.
* sandbox: Reorder result cases in StoreBackedCommandExecutor.
* sandbox: Inject the LedgerTimeAwareCommandExecutor.
* sandbox: Pull out the default time provider type into a constant.
* sandbox: Name ResetService tests consistently.
* sandbox: Reset the time service when resetting in Static Time mode.
The test, unfortunately, is ``@Ignore`d due to flakiness in CI, so won't
actually be run. However, I _hope_ we're going to remove that
annotation eventually, and it allowed me to test-drive the fix on my
machine, so it's still helpful.
CHANGELOG_BEGIN
- [Sandbox] Fix a regression in the ResetService which did not reset the
TimeService in static time mode.
CHANGELOG_END
Turns out not every participant can support seeding (yet).
CHANGELOG_BEGIN
- [Ledger API Server] Re-introduce an option to disable seeding. This
does not affect Sandbox.
CHANGELOG_END
* Set the `Bearer ` prefix in bindings.
* Make the `Bearer ` prefix in the authorization header mandatory.
* Bearer prefix can be removed from the token file.
CHANGELOG_BEGIN
[Extractor]: The ``Bearer `` prefix can be removed from the token file.
It is added automatically.
[Navigator]: The ``Bearer `` prefix can be removed from the token file.
It is added automatically.
[DAML Script] The ``Bearer `` prefix can be removed from the token file. It
is added automatically.
[DAML Repl] The ``Bearer `` prefix can be removed from the token file. It is
added automatically.
[Scala Bindings] The ``Bearer `` prefix can be removed from the token. It is
added automatically.
[Java Bindings] The ``Bearer `` prefix can be removed from the token. It is
added automatically.
[DAML Integration Kit] ``AuthService`` implementations MUST read the
``Authorization`` header and the value of the header MUST start with
``Bearer ``.
CHANGELOG_END
* Produce performance tests for all envelopes
CHANGELOG_BEGIN
[TestTool] Provide performance tests for all performance envelopes
CHANGELOG_END
* Follow Scala's recommended pattern for enum-like hierarchies
There have been reports of sporadic occurrences of
`java.lang.RuntimeException: Lob not found: 49/-2`
when running the participant server with H2.
It seems like using `setBinaryStream` triggers H2 to go through the BLOB
machinery. Since this issue is not easily reproducible, it seems like an
altogether better solution to switch to using `setBytes`. Offsets aren't
that large anyway, so going directly for byte array should be fine.
Better than a broken query anway.
CHANGELOG_BEGIN
CHANGELOG_END
* Add failing test
The test can produce false negatives,
but locally it fails 10 out of 10 times.
* Stop deduplicating commands after rejections
Fixes#5338.
CHANGELOG_BEGIN
CHANGELOG_END
* Only deduplicate successful transactions
* Use pass instead of pure unit
* Handle exceptions
