* Upgrade scala compiler silencer to 1.6.0
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt build bazel file to new targets
* Switch to silencer plugin scala 2.12.11 per Samir's feedback
rather than 2.12.8
* Add missed bazel files
* Review feedback from Leo
* kvutils: Cache state value conversions from bytes.
This seems to have a decent speedup in ledger-on-memory.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Submissions now look up ledger values from a
cache where possible, improving performance when there's contention over
certain resources (e.g. common packages). The cache size currently
defaults to 64 MB.
CHANGELOG_END
* kvutils: Make the SubmissionValidator statue value cache configurable.
* kvutils: Report state value cache metrics.
* kvutils: Add a suffix to a Long literal because WartRemover is unhappy.
Strangely, it doesn't fail on my machine.
* kvutils: Extract caching out into its own file.
* kvutils: Move the `bytesToStateValue` call into `cache.get`.
* kvutils: Move caching to its own package.
* kvutils: Inject the state value cache.
* kvutils: Default to no state value cache.
* kvutils: Accept a state value cache size in megabytes, not bytes.
* kvutils: Move cache building from `Config` to the `caching` package.
* kvutils: Replace Guava's cache with Caffeine.
* kvutils: Simplify caching configuration.
* sandbox: Enable state value caching by default.
CHANGELOG_BEGIN
- [Sandbox] State values deserialization is now cached, with a fixed
cache size of 128MB.
CHANGELOG_END
* Changelog commit.
CHANGELOG_BEGIN
- [Ledger Integration Kit] The state value cache is now opt-in, with a
default of no cache at all.
CHANGELOG_END
* sandbox: Clean up `MetricsReporting` a little.
Make sure it closes both reporters, and avoid starting things in a
constructor.
* sandbox: Add hidden options for enable metrics reporting.
* sandbox: Add a disambiguating name to the DB connection/thread pools.
CHANGELOG_BEGIN
- [Sandbox] DB connection pool metrics names have changed slightly, from
``daml.index.db.connection`` to ``daml.index.db.connection.sandbox``.
- [Ledger Integration Kit] DB connection pool metrics names have changed
to disambiguate the StandaloneApiServer from the
StandaloneIndexerServer. The former now has a ``.ledger-api-server``
suffix, and the latter now has a ``.indexer`` suffix.
CHANGELOG_END
* sandbox-next: Use the same metrics registry for the API and indexer.
* sandbox: Give a useful error message on an invalid metrics reporter.
And simplify the error messages.
With the arguments `--client-auth=foo --metrics-reporter=foo`, we now
get the output:
```
Error: Option --client-auth failed when given 'foo'. Must be one of
"none", "optional", or "require".
Error: Option --metrics-reporter failed when given 'foo'. Must be one of
"console", or "csv:PATH".
Try --help for more information.
```
* sandbox: Pull out more helpers in `MetricsReporting`.
* sandbox: Rename MetricsReporter classes so they don't clash.
* sandbox: Wrap the `name` parameter in a `ServerName` tagged string.
For safety. Yours, not mine.
* sandbox: Push metrics to Graphite with `--metrics-reporter=graphite`.
* sandbox: Make `MetricsReporter.Graphite` singly-lazy, not doubly-.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Replace `ServerName` with `ServerRole`.
* sandbox: Fix usage of `ServerRole.Testing` in `LedgerResource`.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add first prototype of triggers as a service (TaaS)
This is an extremely basic version of the trigger as a service thingy.
Right now, it supports spawning triggers and stopping them but nothing
else.
There is a very simple test to check that it’s not completely broken.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* remove debugging output
* remove leftover import
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* sandbox: Ensure that the Flyway tests actually pick up the SQL files.
* sandbox: Upgrade Flyway from v5 to v6.
CHANGELOG_BEGIN
- [Sandbox] Upgrade the Flyway database migrations library from v5 to v6.
CHANGELOG_END
* ledger-on-sql: Don't bother cleaning up after integration tests.
Turns out Bazel cleans up before each test run, so we only have one
set of outputs at a time. This is far more useful for debugging anyway.
* ledger-on-sql: Pull out a test-lib to simplify the conformance tests.
* ledger-on-sql: Turn `Database` from a sealed trait into a case class.
* ledger-on-sql: Support for PostgreSQL!
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-sql: Run the conformance tests against PostgreSQL.
* ledger-on-sql: Run the LotsOfParties conformance tests.
* ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query.
* ledger-on-sql: Ensure the reader connection pool is read-only.
* ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible.
Instead, use `ByteString#newInput()`, which returns an `InputStream`.
This is supported by H2 and PostgreSQL, but unfortunately not SQLite.
* ledger-on-sql: Run integration tests in parallel.
* Update the PostgreSQL and SQLite JDBC drivers.
* kvutils: Make logback.xml a base file, rather than the only option.
* kvutils/app: Simplify logback.base.xml.
Mostly by getting rid of unnecessary appenders.
* ledger-on-sql: Add trace logging for all database work.
* Upgrade H2; there's a few useful bug fixes.
* kvutils/app: Let the user override the server JDBC URL.
* kvutils/app: Provide a way to specify the ledger ID.
* ledger-on-sql: If there are missing entries in the log, fail on read.
This can happen right now because we insert in parallel. Next step: stop
doing that.
* ledger-on-sql: Stop writing in parallel; it causes race conditions.
This unfortunately means we also stop _reading_ in parallel, which is
less fun.
* Revert "ledger-on-sql: Use a sequential log entry ID."
This reverts commit c58265bf43.
* ledger-on-sql: Create tables with columns that are not nullable.
* ledger-on-sql: The H2 conformance tests now work, just slowly.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-sql: Run tests against H2 and SQLite on memory and disk.
* ledger-on-sql: Allow H2 to read from the log in parallel with writes.
* ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes.
* ledger-on-sql: Make sure to log the correlation ID.
* ledger-on-sql: Do less while holding a database connection.
* ledger-on-sql: Log the connection.
* ledger-on-sql: We don't need to tell H2 not to drop the connection.
The connection pool takes care of that.
* ledger-on-sql: Disable H2 conformance tests on CI; they're too slow.
* ledger-on-sql: Rename `loggingContext` to `logCtx`.
* ledger-on-sql: Don't abuse log contexts; put the data in the message.
* ledger-on-sql: Make the connection log line easier to read.
And pull out the logger, even if it's only used once.
* ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
* Introduce ContextualizedLogger and LoggingContext
A ContextualizedLogger is a wrapper around an Slf4j logger. It uses
call-by-name parameters to only construct logging lines when necessary.
The underlying context can be used by invoking withoutContext.
More interestingly, every call to one of its logging methods requires
the presence of an implicit LoggingContext.
A LoggingContext is a way of definining a set of pairs that a
ContextualizedLogger can log without the need of it being specified for
every call.
A new context can be created with newLoggingContext and, given an
implicit LoggingContext, pairs can be added or overwritten using
withEnrichedLoggingContext.
Pairs in the context will be appended to each event logged by a
ContextualizedLogger in the following form:
some text logged explicitly (context: {key1=value1, key2=value2)
If the underlying Logger is configured to use the Logstash encoder the
keys and values in the LoggingContext will also be added to the logging
event encoded in JSON format.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestion in https://github.com/digital-asset/daml/pull/4046#pullrequestreview-343224692
Thanks to @SamirTalwar
Co-Authored-By: Samir Talwar <samir.talwar@digitalasset.com>
* Address https://github.com/digital-asset/daml/pull/4046#discussion_r366874330
Co-authored-by: Samir Talwar <samir@noodlesandwich.com>
The target override for scalatest was pointing to a meta package that
was empty itself and reexported scalactic and scalatest. It seems this
confused the IntelliJ plugin. This instead introduces dedicated
overrides for scalactic and scalatest.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10
Akka 2.6.1 Upgrade Changes
- Materializer in place of ActorMaterializer
- Source.future instead of Source.fromFuture
- The Scheduler.schedule method has been deprecated in favor of selecting scheduleWithFixedDelay or scheduleAtFixedRate
- onDownstreamFinish(cause: Throwable)
- ActorAttributes.supervisionStrategy(...) in place of ActorMaterializerSettings.withSupervisionStrategy
See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html
* Akka 2.6.1 Upgrade Changes
- onDownstreamFinish(cause: Throwable)
See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html
* code review: remove unnecessary supervision strategy
* Improve Navigator output
* Fix Navigator not using the access token
* Add RSA signatures for JWT tokens
* Remove unused method
* Add timeouts for reading JWKS
* Fix test
* Rename method for consistency
* Improve comment
* More renaming for consistency
* CHANGELOG
CHANGELOG_BEGIN
- [Sandbox] Add CLI options to start the sandbox with JWT based authentication with RSA signed tokens.
See `issue #3155 <https://github.com/digital-asset/daml/issues/3155>`__ .
- [Navigator] Fixed a bug where the `--access-token-file` option did not work correctly.
CHANGELOG_END
* Make JwksVerifier limits configurable
* Make SimpleHttpServer private
* grpc-definitions: Delete health_service.proto
We can use the version in io.grpc:grpc-services instead.
* ledger: Delete ledger/API.md.
* sandbox: Fix warnings in ApiServices flagged by IntelliJ.
* sandbox: Implement a dummy grpc.health.v1.Health.Check endpoint.
* sandbox: Implement a dummy grpc.health.v1.Health.Watch endpoint.
* sandbox: Drop repeated elements from grpc.health.v1.Health.Watch.
* sandbox: Wrap the HealthService in basic tests.
* sandbox: Stop streaming the server health too.
* ledger-api-test-tool: Health check tests.
* Add a changelog entry for the health check endpoints.
CHANGELOG_BEGIN
- [Ledger API] Add healthcheck endpoints, conforming to the
`GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
It is always ``SERVING`` for now.
- [DAML Ledger Integration Kit] Add conformance test coverage for the
``grpc.health.v1.Health`` service.
CHANGELOG_END
* ledger-api-integration-tests: Increment the number of services.
* Apply suggestions from code review
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* sandbox: Use `AkkaBeforeAndAfterAll` in the HealthServiceSpec.
In an attempt to get it working on CI.
* sandbox: Change `dropRepeated` to `DropRepeated()`.
Keep it in one file.
* test-common: Use `Delayed.by` in `TimeBoundObserver`.
* test-common: Close the source when `TimeBoundObserver` completes.
* ./fmt.sh
That'll teach me not to `--no-verify` just because it's a merge commit.
* sandbox: Inline `HealthService.suppress`.
At some point it was being used twice.
* sandbox: Increase the timeout for HealthServiceSpec.
* sandbox: Reimplement HealthService using the Scala protobuf types.
* sandbox: Generate an Akka-compatible trait for the health service.
And refactor a lot of test code to make it easy to test.
* ledger-api-common: Move the HealthService here.
* rs-grpc-testing-utils: Publish to Maven.
* rs-grpc-testing-utils: Add Maven coordinates.
kvutils now supports dumping the ledger to a file via environment
variable: "KVUTILS_LEDGER_DUMP=/tmp/ledger.dump".
The integrity checker tool allows re-processing of ledger dumps to
validate that all submissions, state and log entries can be parsed
and processed. This forms the basis of kvutils data continuity test
suite.
Currently the checker strictly enforces that re-processed
outputs match the inputs exactly, which for now guarantees forwards
and backwards compatibility. The checker will be adapted when
forwards-incompatible changes or new kvutils message versions arise.
A select collection of ledger dumps will be maintained in a separate
repository (to be decided) and these dumps are to be validated by
the DAML CI.
* Update bazel-common to fix javadoc issues
Specifically, to fix the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
javadoc_library(name = 'rs-grpc-bridge_javadoc')
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```
* Define Maven deps using rules_jvm_external
* Pin artifacts
* Remove bazel-deps generated targets
* Remove bazel-deps
* Switch to rules_jvm_external targets
* update bazel documentation
* pom_file: There are no more bazel-deps targets
* BAZEL-JVM.md `maven_install` typo
