Commit Graph

30 Commits

Author SHA1 Message Date
fabiotudone-da
2fd200fff8
KV: support contextual logging in KeyValueConsumption [KVL-1143] (#11288)
* Support contextual logging in KeyValueConsumption

CHANGELOG_BEGIN
CHANGELOG_END

* Fix formatting and imports

* Propagate the contextual logging per-readUpdates call

* No need for at-construction logging context in LogAppendingReadServiceFactory

* No need for logging context in CommitStrategySupport.newReadServiceFactory

* Fix JdbcIndexerSpec
2021-10-19 20:16:37 +00:00
tudor-da
29c546c949
[Divulgence pruning] Added prune_all_divulged_contracts to PruneRequest [DPP-534] (#10635)
* Added `prune_all_divulged_contracts` to ParticipantPruningService.Prune
* KeyValueParticipantStateWriter.prune returns UNIMPLEMENTED if called with `pruneAllDivulgedContracts` set

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments

* Apply suggestions from code review

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-08-23 20:52:27 +00:00
nicu-da
99f59007bf
Add flag to enable/disable command deduplication [KVL-1006] (#10518)
* Add flag to enable/disable command deduplication

* Remove flags for configuration as it should not be exposed externally

* Move deduplicationEnabled flag to the write service.

The deduplication enabled flag is tightly coupled to the WriteService implementation so the flag has been moved to the WriteService trait so that it's explicitly defined.

CHANGELOG_BEGIN
Command deduplication is enabled/disabled based on the write service implementation.
v1 WriteService enables command deduplication while v2 WriteService disabled it
CHANGELOG_END

* Rename deduplication flag
Set sandbox deduplication to true and proxy write service to the delegage
2021-08-11 09:27:25 +02:00
Samir Talwar
b325e8a8b8
participant-state: Remove WriteService#rejectSubmission. (#10407)
It's not currently used. We will re-introduce it when we plan on
actually implementing it.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-26 13:03:05 +00:00
Samir Talwar
b9518ce2f4
participant-state-metrics: Add wrappers for v2. (#10404)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-26 11:06:02 +00:00
Samir Talwar
cf2b61e6ac
participant-state: Remove the aliases to Ref.LedgerString. (#10325)
* participant-state: Remove the aliases to `Ref.LedgerString`.

Use the aliases directly in `Ref` instead.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/data: Improve comments on ID aliases.

* daml-lf/data: Fix a typo in a comment.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-21 08:34:46 +00:00
Samir Talwar
9a01065f29
participant-state: Remove the Daml-LF transaction and value aliases. [KVL-1002] (#10317)
* participant-state: Remove the Daml-LF transaction aliases.

They add nothing.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Move the `ContractInst` alias to where it's used.

Either inlining it or extracting out a similar alias elsewhere.

This further removes dependencies on _participant-state_.

* Fix a couple of typos.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-19 18:32:28 +00:00
Samir Talwar
bd12170af2
participant-state: Remove the ParticipantId, PackageId, and Party aliases. [KVL-1002] (#10308)
* participant-state: Remove the `ParticipantId` alias.

This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Remove the `PackageId` and `Party` aliases.

They don't add anything. Let's just use `Ref`.

* kvutils: Restore missing compat imports.
2021-07-19 12:31:25 +00:00
Samir Talwar
cb29f34d4b
ledger-offset: Move Offset to a new package. [KVL-1002] (#10296)
* ledger-offset: Move `Offset` to a new package.

CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
  package, ``ledger-offset``, from the ``participant-state`` package.
  The Java package has been renamed to ``com.daml.ledger.offset``. If
  you are using this type, you will need to update your dependencies and
  imports.
CHANGELOG_END

* Avoid rewrapping offsets for no reason.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* participant-integration-api: Sort some imports.

* participant-integration-api: Fix dependencies for the Oracle tests.

I didn't add `ledger-offset`.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 17:53:03 +02:00
Samir Talwar
a9a0b70b85
ledger-configuration: Extract the configuration from participant-state. [KVL-1002] (#10279)
* ledger-configuration: Extract configuration from participant-state.

The configuration is often used without the state, and doesn't need to
be versioned in the same way.

CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
  ``LedgerInitialConditions``, and ``TimeModel``, have been moved from
  *participant-state* to a separate package named
  *ledger-configuration*, in the Java package
  ``com.daml.ledger.configuration``. You will need to update your
  dependencies and imports.
CHANGELOG_END

* participant-state: Remove the `LedgerId` aliases.

* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.

This avoids confusion with the protobuf-generated `TimeModel` classes.

CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
  ``LedgerTimeModel``. If you are using the ledger configuration classes
  directly, you may need to update your code.
CHANGELOG_END

* ledger-configuration: Remove colons in LedgerInitialConditions' docs.

* kvutils: Restore a missing compat import.

* participant-integration-api: Add ledger-configuration to Oracle tests.

* sandbox-common: Fix `--max-ledger-time-skew` docs.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 13:03:19 +00:00
Hubert Slojewski
bef8af2bb6
Propagate trace context for configuration submission (#9545)
CHANGELOG_BEGIN
CHANGELOG_END
2021-05-04 09:38:56 +02:00
Hubert Slojewski
0e5c4d4782
KVL-919 Propagate trace context for package upload (#9543)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-30 16:51:25 +02:00
Hubert Slojewski
ceec40b299
Propagate trace context for party allocation (#9524)
CHANGELOG_BEGIN
CHANGELOG_END
2021-04-29 09:42:15 +02:00
Hubert Slojewski
f4316bddf3
KVL-874 Add telemetry classes from the oem integration kit and use it for command submissions (#9436)
* Add telemetry classes from the oem integration kit and use it for command submissions

* Change submitTransaction to submitTransactionWithTelemetry and add a deprecation

* Fix tests

* Revert "Change submitTransaction to submitTransactionWithTelemetry and add a deprecation"

CHANGELOG_BEGIN

- [Integration Kit] TelemetryContext has been introduced to the WriteService.submitTransaction method to support distributed tracing

CHANGELOG_END
2021-04-23 13:00:39 +02:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Moritz Kiefer
424faa923a
Port damlc dependencies to Scala 2.13 (#8423)
* Port damlc dependencies to Scala 2.13

I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).

So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).

If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.

changelog_begin
changelog_end

* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* fixup lawlesstraversal

changelog_begin
changelog_end

* less iterator more view

changelog_begin
changelog_end

* document safety of unsafeWrapArray

changelog_begin
changelog_end

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-01-08 07:22:38 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Samir Talwar
9976b4cd50
Bazel: Factor out logic around Protobuf JARs. [KVL-714] (#8084)
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.

* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.

* Bazel: Factor out Java/Scala protobuf class generation into a helper.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/archive: Use `proto_jars`.

* Bazel: Remove the visibility modifier from `proto_jars`.

It's too confusing. Just make everything public.

* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.

* Bazel: Add comments to the various parts of `proto_jars`.

* daml-assistant: Do unpleasant things with `location` in Bazel.
2020-11-27 08:34:53 +00:00
Oliver Seeliger
83f11aa784
Participant pruning ledger api server support ported from Canton (#7988)
* Participant pruning ledger api server support ported from Canton

CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END

* Review feedback from Stefano

* Add pruning tests plus missed command completions change

* Review feedback from Robert

* Improved test readability by having populate helper return offsets

* Review feedback

* Ledger api changes to pruning api and disable canton pruning test

- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string

* Review feedback: Use ApiOffsetSConverter for logged offsets
2020-11-23 15:28:04 +01:00
Robert Autenrieth
0c1cadcfc4
Add metrics for concurrent commands [KVL-583] (#7575)
* Add metrics for concurrent commands

* Update readme

CHANGELOG_BEGIN
- [DAML on SQL] Add new metrics for measuring the number
  of concurrent command executions. The metrics are:
  daml.commands.submissions_running, daml.execution.total_running,
  daml.execution.engine_running
CHANGELOG_END
2020-10-06 21:09:54 +02:00
Miklos
ccfb0ac94e
Make interpretation cost available for LedgerWriter/WriteService (#6515)
* Measure time of interpretation and store it in CommandExecutionResult.

* Added parameters for passing in interpretation time for WriteService and LedgerWriter.

* Code tidying.

* Added CommitMetadata parameter to LedgerWriter.

* Thread through interpretation time. Aggregate interpretation times for a batch.

* Take max of interpretation times.

* Calculate interpretation time in nanos.

* Moved CommitMetadata into separate file.
CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Delegate deprecated methods to method with new signature.

* Code tidying.

* Suppress deprecation warnings.

* Made interpretation cost optional in CommitMetadata.

* Test that we populate interpretation time.

* Code tidying.

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Code tidying.

* Some more tests. Do not return interpretation cost for a batch if it only contains non-transaction submissions.

* Reformatted.

* Reformatted.

* Named arbitrary interpretation cost.

* Reverted changes for BatchingLedgerWriter.

* Always drop commit metadata for batches and don't report interpretation cost.

* More specific expectations.

* Include authorization check and blinding in interpretation time.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-06-30 16:45:12 +00:00
Gerolf Seitz
4d606689bb
Centralize metric names (#5790)
Centralizing the metrics allows us to have a good overview of the
existing metrics.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-05 16:55:29 +02:00
Samir Talwar
bdb476fff0
ledger/metrics: Move metric helpers to their own Bazel package. (#5542)
* ledger/metrics: Move metric helpers to their own Bazel package.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Use ledger/metrics.

* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.

Importing methods is harder than importing objects.

* metrics: Publish to Maven Central.
2020-04-14 12:32:03 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Stefano Baghino
c23564b0cc
Use snake_case for all metrics (#5348)
* Use snake_case for all metrics

This makes sure we're using the snake_case for all metrics. DAML-on-X originally asked us for this.

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/5348#issuecomment-607326336
2020-04-01 18:53:02 +00:00
Samir Talwar
fca8895954
kvutils | sandbox: Wrap metric names in a value type. (#5306)
* participant-state-metrics: Wrap metric names in a value type.

For safety, and for simplicity when building upon prefixes.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Use MetricName within MetricsNaming.

* kvutils: Use `MetricName` in `Committer`.

* sandbox | kvutils: Extract common metric prefixes.

* sandbox: Remove a redundant visibility modifier in `MetricsNaming`.

* participant-state-metrics: `MetricName` doesn't need to be a case class.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-04-01 15:35:28 +00:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Samir Talwar
66a7fc6384
kvutils/app: Publish metrics someplace. (#5267)
* kvutils/app: Publish JVM metrics.

* kvutils/app: Sort the metrics prefixes.

* kvutils/app: Allow for specifying a metrics reporter as a CLI argument.

No changelog as this is hidden.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Remove an unnecessary string interpolation mark.

Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2020-03-30 13:00:42 +00:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Samir Talwar
69220c3f58
kvutils/app: Add timing metrics for read/write/index services. (#5176)
* participant-state{,-index}: Move Timed*Service classes from Sandbox.

CHANGELOG_BEGIN
- [Ledger Integration Kit] Metrics for the various read, write, and index
  services.
CHANGELOG_END

* kvutils/app: Add timing metrics for read/write/index services.

* participant-state: Move metrics-related code to another Bazel package.

* participant-state-metrics: Add to artifacts.yml.

* participant-state-metrics: Move TimedIndexService back into Sandbox.

Cuts down on dependencies like nobody's business.
2020-03-25 15:11:37 +00:00