Commit Graph

115 Commits

Author SHA1 Message Date
Stefano Baghino
6f1e051648
Integrate metrics for cache (#6043)
* Integrate metrics for cache

Centralizes the creation of instrumented caches in a single point and adds
metrics coming from Caffeine into the mix.

changelog_begin
[Ledger API Server] if --max-state-value-cache-size is greater than zero, the
following additional metrics will be recorded under the daml.kvutils.submission.validator.state_value_cache namespace:
hits, misses, load_successes, load_failures, load_total_time, evictions and evicted_weight
changelog_end

* Fix Bazel build file formatting

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427902339

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427904794

* Review DropwizardStatsCounter

- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905074
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905184
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905307
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905387
- address https://github.com/digital-asset/daml/pull/6043#discussion_r427905650

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427906243

* Fix implicit numeric widening fatal warning

* Address https://github.com/digital-asset/daml/pull/6043#discussion_r427960762

* Fix infinite loop in metrics
2020-05-20 15:51:45 +02:00
Samir Talwar
8cd3cc414e
ledger/metrics: Display graphs with counts alongside mean timings. (#5994)
CHANGELOG_BEGIN
CHANGELOG_END
2020-05-15 08:44:34 +00:00
Stefano Baghino
b0ad8d7a60
Centralize and time serialization (#5946)
changelog_begin
OVERRIDES CHANGELOG ENTRY FROM 2a8c93a614
[Ledger Integration Kit] Added new metrics for
``daml.index.db.*.translation`` to measure the time spent
translating to and from the serialized DAML-LF values when
fetched from the participant index.
changelog_end
2020-05-13 17:06:46 +02:00
Stefano Baghino
dd6af4e276
Centralize index DB metrics naming (#5927)
* Centralize and review index DB metrics naming

Closes #5922

changelog_begin
changelog_end

* Remove unnecessary factory

* Fix compilation issue
2020-05-11 12:59:02 +00:00
Stefano Baghino
2a8c93a614
Offload value deserialization from DB threadpool and add metrics (#5917)
* Offload value deserialization from DB threadpool and add metrics

Fixes #5909

CHANGELOG_BEGIN
[Ledger Integration Kit] Added new metrics for
``daml.index.db.*.deserialization`` to measure the duration of
the translation of the serialized DAML-LF values when fetched
from the participant index.
CHANGELOG_END

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422282582

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422282901

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422283768

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422285920

* Address https://github.com/digital-asset/daml/pull/5917#discussion_r422285674
2020-05-11 10:41:49 +02:00
Robert Autenrieth
9ebfd01f83
Add script to dump metrics summary (#5828)
* Add script to dump metrics summary

CHANGELOG_BEGIN
CHANGELOG_END

* Sort results by default

* Address review comments

* Run prettier

* Refactor code

* Make script executable

* Address review comment

* Always take latest data point
2020-05-06 17:13:14 +02:00
Gerolf Seitz
4d606689bb
Centralize metric names (#5790)
Centralizing the metrics allows us to have a good overview of the
existing metrics.

CHANGELOG_BEGIN
CHANGELOG_END
2020-05-05 16:55:29 +02:00
Miklos
d6c5a2532d
Add counting future metrics (#5743)
* Added counting metrics.

* Code tidying.
CHANGELOG_BEGIN
CHANGELOG_END

* Rename based on review suggestion.
2020-04-28 13:58:15 +00:00
Samir Talwar
21ee16623e
sandbox: Revert execution metric paths. (#5588)
Some customers rely on these.

CHANGELOG_BEGIN
- [Ledger API Server] Moved the ``daml.commands.execution`` back to
  ``daml.execution``.
CHANGELOG_END
2020-04-16 18:30:29 +00:00
Samir Talwar
e854191586
kvutils: Add a timing metric for the commit. (#5589)
CHANGELOG_BEGIN
- [Ledger Integration Kit] Added a timing metric for the commit at
  ``daml.kvutils.writer.commit``.
CHANGELOG_END
2020-04-16 18:30:09 +00:00
Samir Talwar
6a1b8a337f
ledger-api-common: Add a metric for command validation upon submission. (#5585)
* ledger-api-common: Add a metric for command submission validation.

CHANGELOG_BEGIN
- [Ledger API Server] Add a metric for command validation upon
  submission, ``daml.commands.validation``.
CHANGELOG_END

* ledger-api-common: Move the submissions metric from sandbox.
2020-04-16 16:46:16 +00:00
Samir Talwar
02eadf68c8
sandbox: Moar command submission metrics. (#5583)
* sandbox: Add a test for MetricsInterceptor for non-streaming endpoints.

* sandbox: Simplify MetricsInterceptor by overriding `ServerCall`.

* sandbox: Time all submissions, not just through the submission service.

* sandbox: Don't bother timing transaction submissions twice.

They're already timed by TimedWriteService.

* sandbox: Move command submission metrics to "daml.commands".

CHANGELOG_BEGIN
- [Ledger API Server] A number of command submission metrics have moved,
  and we have added a couple.
  - ``daml.commands.submissions`` is a new timer that measures all
    submissions.
  - ``daml.commands.valid_submissions`` is a new meter that counts
    valid (unique, interpretable) submissions.
  - ``daml.lapi.command_submission_service.failed_command_interpretations``
    has been moved to ``daml.commands.failed_command_interpretations``.
  - ``daml.lapi.command_submission_service.deduplicated_commands`` has
    been moved to ``daml.commands.deduplicated_commands``.
  - ``daml.lapi.command_submission_service.delayed_submissions`` has
    been moved to ``daml.commands.delayed_submissions``.
  - ``daml.lapi.command_submission_service.submitted_transactions`` has
    been moved to ``daml.services.write.submit_transaction``.
  - All ``daml.execution.*`` metrics have moved to
    ``daml.commands.execution``.
CHANGELOG_END

* metrics: Update the dashboard to include the new command metrics.
2020-04-16 15:20:39 +00:00
Samir Talwar
9186f5ccb6
kvutils | ledger-on-(memory|sql): Read metrics. (#5576)
* ledger-on-memory: Add metrics to the operations.

* ledger-on-memory: Speed up reads by using views.

* kvutils: Time event processing.

* ledger-on-(memory|sql): Make classes private and final where possible.

* kvutils: Factor out a TimedLedgerStateOperations class.

CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
  operations.
CHANGELOG_END

* ledger-on-sql: Record log read metrics.

* ledger-on-sql: Record database transaction timing metrics.

CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END

* ledger-on-sql: Time queries.

* metrics: Add graphs for read events and DAML-on-SQL.

* ledger-on-memory: Simplify the tuple swap in reading the log.

Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2020-04-16 11:11:06 +00:00
Samir Talwar
5c1c7fe30a
metrics: Collect with Graphite and visualize with Grafana. (#5549)
* Add compose config for grafana/graphite combo

* Add rudimentary readme with instructions

* Add Samir's dashboard

* metrics: Move `metrics/grafana` to `ledger/metrics/collection`.

CHANGELOG_BEGIN
CHANGELOG_END

* metrics/collection: Rename samirs-dashboard to ledger-submissions.

* metrics/collection: Save Graphite data between restarts.

* metrics/collection: Add copyright headers.

Co-authored-by: Vladislav Gangan <vladislav.gangan@digitalasset.com>
2020-04-15 12:28:57 +00:00
Samir Talwar
bdb476fff0
ledger/metrics: Move metric helpers to their own Bazel package. (#5542)
* ledger/metrics: Move metric helpers to their own Bazel package.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Use ledger/metrics.

* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.

Importing methods is harder than importing objects.

* metrics: Publish to Maven Central.
2020-04-14 12:32:03 +00:00