Commit Graph

12 Commits

Author SHA1 Message Date
Robert Autenrieth
f058c2f16f
DPP-368 clean up flags (#10711)
* Make batching parallelism configurable

changelog_begin
changelog_end

* Fail if incompatible cli flags are used
2021-08-31 23:43:32 +00:00
Samir Talwar
733590d591
ledger-api-health: Use the Scala health status values everywhere. (#10640)
The methods are intended to be used by Java code, where Scala
`case object` values can be unidiomatic.

CHANGELOG_BEGIN
CHANGELOG_END
2021-08-20 16:50:20 +00:00
Kamil Bozek
3b1d1ac7ac
IndexerBenchmark: an option for defining minimum update rate [DPP-541] (#10540)
* Changed default metrics reporting interval to 1 second

* Min update rate SLO

* Return exit code 1 on benchmark failure

* CHANGELOG_BEGIN
- [Integration Kit] - indexer-benchmark - added --min-update-rate option for defining required update rate
CHANGELOG_END
2021-08-10 23:08:06 +02:00
Samir Talwar
d666f76fbb
participant-integration-api: Switch to the v2 participant state API. [KVL-1002] (#10398)
* participant-state: Give `ChangeId` its own file.

* participant-state: Introduce `v1.CompletionInfo` for a while.

`CompletionInfo` is identical to `SubmitterInfo`. Its purpose is to make
the migration to v2 easier. It should not make it into the final
version.

* participant-integration-api: Switch to the v2 participant state API.

This means that:

  - the API server and indexer expect v2 API traits
    - adapters are provided if you need to elevate your v1 API usage
  - the indexer internally uses v2 Updates
  - rejections are switched over to use the v2 format
  - Sandbox Classic uses v2 as the internal representation too (because
    it interacts directly with the underlying indexer representation,
    and is therefore tightly coupled)

_kvutils_ and other users of the `StandaloneApiServer` and
`StandaloneIndexerServer` use the adapters.

CHANGELOG_BEGIN
- [Integration Kit] The API server and indexer have switched over to v2
  of the participant-state API. You can continue to use the v1 API, but
  you will need to wrap your ``ReadService`` and ``WriteService``
  objects in the ``AdaptedV1ReadService`` and ``AdaptedV1WriteSerivce``
  classes.
CHANGELOG_END

* participant-state: Remove v1.CompletionInfo.

It's served its purpose.

* kvutils: Remove an unnecessary line from `Runner`.

* ledger-api-common: Delete a TODO; we'll track it elsewhere.

* participant-integration-api: Use full words in `JdbcLedgerDao`.

Just a little bit of cleanup.

* ledger-api-common: Extract out the random submission ID generator.

And introduce a trait, because, well, this is still the JVM.
2021-07-28 14:59:23 +00:00
Kamil Bożek
8b337bda4b
Publish ledger-indexer-benchmark (#10401)
CHANGELOG_BEGIN
CHANGELOG_END
2021-07-26 13:53:56 +02:00
Samir Talwar
c3bc26fd1c
daml-lf/data: Move ID aliases to Ref from _ledger-api-common_. [KVL-1002] (#10323)
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.

This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-integration-api: Remove an unused import.

* http-json-oracle: Remove `ledger-api-common` as a dependency.

* bindings-rxjava: Remove a now-unused dependency.
2021-07-20 11:01:19 +00:00
Samir Talwar
cb29f34d4b
ledger-offset: Move Offset to a new package. [KVL-1002] (#10296)
* ledger-offset: Move `Offset` to a new package.

CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
  package, ``ledger-offset``, from the ``participant-state`` package.
  The Java package has been renamed to ``com.daml.ledger.offset``. If
  you are using this type, you will need to update your dependencies and
  imports.
CHANGELOG_END

* Avoid rewrapping offsets for no reason.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* participant-integration-api: Sort some imports.

* participant-integration-api: Fix dependencies for the Oracle tests.

I didn't add `ledger-offset`.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 17:53:03 +02:00
Samir Talwar
a9a0b70b85
ledger-configuration: Extract the configuration from participant-state. [KVL-1002] (#10279)
* ledger-configuration: Extract configuration from participant-state.

The configuration is often used without the state, and doesn't need to
be versioned in the same way.

CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
  ``LedgerInitialConditions``, and ``TimeModel``, have been moved from
  *participant-state* to a separate package named
  *ledger-configuration*, in the Java package
  ``com.daml.ledger.configuration``. You will need to update your
  dependencies and imports.
CHANGELOG_END

* participant-state: Remove the `LedgerId` aliases.

* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.

This avoids confusion with the protobuf-generated `TimeModel` classes.

CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
  ``LedgerTimeModel``. If you are using the ledger configuration classes
  directly, you may need to update your code.
CHANGELOG_END

* ledger-configuration: Remove colons in LedgerInitialConditions' docs.

* kvutils: Restore a missing compat import.

* participant-integration-api: Add ledger-configuration to Oracle tests.

* sandbox-common: Fix `--max-ledger-time-skew` docs.

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-07-15 13:03:19 +00:00
Victor Peter Rouven Müller
4037b1cd9e
Add metrics to the http json service (#9923)
* Introduce metrics in the http-json service

changelog_begin

[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)

changelog_end

* Move MetricsReporter and it's dependencies into //ledger/metrics

* Restore non-ugly formatting for that one section in Endpoints.scala

* Update ledger/sandbox-common/src/test/lib/scala/platform/sandbox/cli/CommonCliSpecBase.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Hide metrics option for http-json

* Propagate context exception for the parseUri function in MetricsReporter.scala

* Move cliHint value above parseUri function to have a better structure (it's used once before it's defined and once after it's defined, which is weird to me)

* Use better value name than optMr & optFd in cliopts Metrics.scala

* Remove import order changes & whitespace changes

* Revert usage of Nanoseconds for conversion from scala duration to java duration to usage of Seconds

* Shorten hideIfRequested function

* Fix another rearranged import

* Fix another whitespace removal

* Readd metrics cli option to sandbox after refactoring

* Add missing return type annotation for invalidRead in MetricsReporter

* Readd newline in https OptionParser.scala

* Remove unecessary import

* Update ledger/sandbox-common/src/main/scala/platform/sandbox/cli/CommonCliBase.scala

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* Align setter & config name for metricsReportingInterval setting too in CommonCliBase.scala

* Rename http_json_api in Metrics.scala of metrics project to HttpJsonApi

* Reformat CommonCliBase.scala of sandbox-common project

* Fix CommonCliSpecBase test of sandbox

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-06-08 15:54:01 +00:00
Victor Peter Rouven Müller
beb2d138f2
Add cli option & system property to enable json only logging for http json service (#9725)
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
2021-05-20 13:28:34 +00:00
Marton Nagy
e50885caa8
Dpp 388 todo cleanup parallel indexer (#9690)
* Removes TODO related to BatchingParallelIngestionPipe type: the result of the ingestion is not needed at the moment
* Removes TODO related to groupedWithin: at this point the introduced latency seems not significant
* Refactor AsyncSupport
* Removing some TODOs related to postponed work-items
* moving mutable initialisation code does not make sense without refactoring/segregating the initialisation part of the LedgerDao. That effort is postponed as per group decision.
* Injecting the dispatcher makes little sense at this point, further features might require it.
* Fixes type derivation in ParallelIndexerFactory
* Moves instrumentedBufferedSource to metrics (+ unit tests)
* Removes TODO related to logging: primarily to pass down the initializing LoggingContext is fine. (further logging discussion is in progress related to ingestion at the time of writing)
* Refactor type cast towards synthetic zero-element generation
* Expose configuration for the instrumented input, lowers default.
* Remove TODO related wiring of indexer HealthChecks: there is already an issue for this: https://github.com/digital-asset/daml/issues/2733
* Remove TODO related to lowering connectionTimeout for Hikari database-connection pool: it is already the minimum, no smaller connection timeout is available for selection.

changelog_begin
changelog_end
2021-05-18 02:34:02 +02:00
Robert Autenrieth
03034ec3bf
DPP-359 Add indexer flow level benchmark (#9509)
* Add indexer flow level benchmark

changelog_begin
changelog_end

* Add readme

* Address review comments

* Fix scala 2.13 build

* Small improvements

* fix format errors
2021-05-06 15:43:34 +00:00