Commit Graph

61 Commits

Author SHA1 Message Date
Samir Talwar
1ce67fbd2f
kvutils: Metrics on the write path. (#5212)
* kvutils: Add metrics to SubmissionValidator.

* kvutils: Add metrics to KeyValueSubmission.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-26 15:17:42 +00:00
Samir Talwar
2887f28cc1
kvutils: Inject the metrics registry instead of using SharedMetricRegistries. (#5161)
* kvutils: Remove an unnecessary `@SuppressWarnings`.

* kvutils: Reduce the scope of fields and methods in `Committer`.

* kvutils: Inject the metric registry into `KeyValueCommitting`.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Inject the metric registry into the committers.

* kvutils: Inject the metric registry into `ProcessTransactionSubmission`.

* kvutils: Avoid shared metric registries in tests.

* kvutils: Recreate the metrics registry per participant state.

* kvutils: Add trailing commas to parameter lists.

Flagrantly encouraged by @stefanobaghino-da.

* recovering-indexer: Don't re-use the metric registry in tests.
2020-03-24 19:41:19 +00:00
Remy
8fb86b7133
Engine: create type for immutable byte array (#5140)
CHANGELOG_BEGIN
CHANGELOG_END
2020-03-24 09:04:52 +01:00
Samir Talwar
3d6ecc13b6
Sandbox: Configurable metrics output. (#5113)
* sandbox: Clean up `MetricsReporting` a little.

Make sure it closes both reporters, and avoid starting things in a
constructor.

* sandbox: Add hidden options for enable metrics reporting.

* sandbox: Add a disambiguating name to the DB connection/thread pools.

CHANGELOG_BEGIN
- [Sandbox] DB connection pool metrics names have changed slightly, from
  ``daml.index.db.connection`` to ``daml.index.db.connection.sandbox``.
- [Ledger Integration Kit] DB connection pool metrics names have changed
  to disambiguate the StandaloneApiServer from the
  StandaloneIndexerServer. The former now has a ``.ledger-api-server``
  suffix, and the latter now has a ``.indexer`` suffix.
CHANGELOG_END

* sandbox-next: Use the same metrics registry for the API and indexer.

* sandbox: Give a useful error message on an invalid metrics reporter.

And simplify the error messages.

With the arguments `--client-auth=foo --metrics-reporter=foo`, we now
get the output:

```
Error: Option --client-auth failed when given 'foo'. Must be one of
  "none", "optional", or "require".
Error: Option --metrics-reporter failed when given 'foo'. Must be one of
  "console", or "csv:PATH".
Try --help for more information.
```

* sandbox: Pull out more helpers in `MetricsReporting`.

* sandbox: Rename MetricsReporter classes so they don't clash.

* sandbox: Wrap the `name` parameter in a `ServerName` tagged string.

For safety. Yours, not mine.

* sandbox: Push metrics to Graphite with `--metrics-reporter=graphite`.

* sandbox: Make `MetricsReporter.Graphite` singly-lazy, not doubly-.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Replace `ServerName` with `ServerRole`.

* sandbox: Fix usage of `ServerRole.Testing` in `LedgerResource`.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-03-23 10:59:13 +00:00
Samir Talwar
28c78e5978
sandbox | kvutils/app: Re-use the Materializer in the JdbcIndexer. (#5124)
* kvutils/app|sandbox: Rename `system` to `actorSystem`.

* sandbox: Pass a materializer in to the StandaloneIndexerServer.

There's no need for it to construct one when the caller always has one
available.

CHANGELOG_BEGIN
CHANGELOG_END

* recovering-indexer: Use `SubmissionId` instead of `LedgerString`.

Where appropriate.

* sandbox: Use the materializer implicitly in JdbcIndexer.
2020-03-23 08:55:19 +00:00
Samir Talwar
1ffd23a6a2
Sandbox: Make JdbcLedgerDao create its own execution context. (#5091)
* sandbox: Don't let just anyone construct a DbDispatcher.

It's the job of the JdbcLedgerDao.

* sandbox: Clean up the DbDispatcher a little.

* sandbox: JdbcLedgerDao now creates its own execution context.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Make `defaultNumberOfShortLivedConnections` private.
2020-03-20 15:30:35 +00:00
Samir Talwar
b9acad2c6d
kvutils: Don't supply execution contexts to ResourceOwners. (#5010)
* kvutils: Remove the unnecessary execution context from the test base.

* kvutils: Remove the unnecessary execution context from the writer.

* ledger-on-sql: Make a proper owner so it has a proper execution context.

This means the parallelization now needs to come from the test, so I've
augmented ParticipantStateIntegrationSpecBase to take a proper execution
context instead of the serial one that ScalaTest provides, with a
default of `ExecutionContext.global`.

* ledger-on-memory: Make a proper owner with a proper execution context.

* kvutils/app: Remove `executionContext` from LedgerFactory.

Shouldn't need it in `ResourceOwner`. I was bad.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-memory: Make ResourceOwners real classes.

* ledger-on-sql: Make the ResourceOwner a real class.

* ledger-on-sql: Cause side effects on resource acquisition.

Not on owner construction.
2020-03-16 19:05:23 +00:00
Samir Talwar
3e7d016875
sandbox: Normalize language around listing all known parties. (#4895)
This renames methods backing the `ListKnownParties` request to
from `parties`, `getParties` or `listParties` to `listKnownParties`.

CHANGELOG_BEGIN
- [Ledger API Server] Renamed two metrics:
  ``daml.index.parties`` was renamed to ``daml.index.list_known_parties``
  ``daml.index.db.get_parties`` was renamed to ``daml.index.db.list_known_parties``
CHANGELOG_END
2020-03-09 13:15:33 +00:00
mergify[bot]
e785bb7402
Sandbox: Re-use actor systems between the StandaloneApiServer and StandaloneIndexerServer. (#4827)
* sandbox: Re-use the root actor system in the StandaloneIndexerServer.

* kvutils/app: Don't use the ActorSystem execution context randomly.

Instead, make `Runner` a proper ResourceOwner, with an `acquire` method.

* sandbox: Re-use the root actor system in the StandaloneApiServer.

CHANGELOG_BEGIN
CHANGELOG_END

* resources: Remove the now-unused `ResourceOwner.sequence` functions.

They weren't well-thought-out anyway; they acquire resources
sequentially, rather than in parallel.
2020-03-05 14:50:18 +00:00
Gerolf Seitz
d617922618
Remove InMemoryKVParticipantState (#4674)
This removes the sample/reference implementation of kvutils
InMemoryKVParticipantState.
This used to be the only implementation of kvutils, but now with the
simplified kvutils api we have ledger-on-memory and ledger-on-sql.

InMemoryKVParticipantState was also used for the ledger dump utility,
which now uses ledger-on-memory.

* Runner now supports a multi participant configuration
This change removes the "extra participants" config and goes for consistent
participant setup with --participant.

* Run all conformance tests in the repository in verbose mode.

This means we'll print stack traces on error, which should make it
easier to figure out what's going on with flaky tests on CI.

This doesn't change the default for other users of the
ledger-api-test-tool; we just add the flag for:

  - ledger-api-test-tool-on-canton
  - ledger-on-memory
  - ledger-on-sql
  - sandbox




Fixes #4225.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-26 15:45:35 +01:00
Samir Talwar
d729e5135e
recovering-indexer-integration-tests: Extract from reference-v2. (#4636)
This moves IndexerIT into its own package, and swaps the dependency from
reference-v2 to ledger-on-memory.

This test should ideally live in the sandbox code, but because it
depends on ledger-on-memory, it's easier to keep it separate.

Also rewrites a lot of the code because the API is different. The tests
should now be clearer too.

I've also marked the test as flaky, because, well, it is.

CHANGELOG_BEGIN
CHANGELOG_END
2020-02-21 11:10:32 +00:00