Commit Graph

17 Commits

Author SHA1 Message Date
Moritz Kiefer
6453a73968
Add first prototype of triggers as a service (TaaS) (#4287)
* Add first prototype of triggers as a service (TaaS)

This is an extremely basic version of the trigger as a service thingy.

Right now, it supports spawning triggers and stopping them but nothing
else.

There is a very simple test to check that it’s not completely broken.

changelog_begin
changelog_end

* Apply suggestions from code review

Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

* remove debugging output

* remove leftover import

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-01-31 12:53:45 +01:00
Samir Talwar
2737303ac1
sandbox: Upgrade Flyway from v5 to v6. (#4207)
* sandbox: Ensure that the Flyway tests actually pick up the SQL files.

* sandbox: Upgrade Flyway from v5 to v6.

CHANGELOG_BEGIN
- [Sandbox] Upgrade the Flyway database migrations library from v5 to v6.
CHANGELOG_END
2020-01-24 19:36:31 +01:00
Samir Talwar
0be3e72a12
ledger-on-sql: PostgreSQL support. (#4204)
* ledger-on-sql: Don't bother cleaning up after integration tests.

Turns out Bazel cleans up before each test run, so we only have one
set of outputs at a time. This is far more useful for debugging anyway.

* ledger-on-sql: Pull out a test-lib to simplify the conformance tests.

* ledger-on-sql: Turn `Database` from a sealed trait into a case class.

* ledger-on-sql: Support for PostgreSQL!

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Run the conformance tests against PostgreSQL.

* ledger-on-sql: Run the LotsOfParties conformance tests.

* ledger-on-sql: Use PostgreSQL's `RETURNING` keyword to save a query.

* ledger-on-sql: Ensure the reader connection pool is read-only.

* ledger-on-sql: Avoid cloning ByteStrings as byte arrays where possible.

Instead, use `ByteString#newInput()`, which returns an `InputStream`.
This is supported by H2 and PostgreSQL, but unfortunately not SQLite.

* ledger-on-sql: Run integration tests in parallel.

* Update the PostgreSQL and SQLite JDBC drivers.
2020-01-24 15:33:11 +01:00
Samir Talwar
2e3c6b28bd
ledger-on-sql: Get H2 working in the conformance tests. (#4155)
* kvutils: Make logback.xml a base file, rather than the only option.

* kvutils/app: Simplify logback.base.xml.

Mostly by getting rid of unnecessary appenders.

* ledger-on-sql: Add trace logging for all database work.

* Upgrade H2; there's a few useful bug fixes.

* kvutils/app: Let the user override the server JDBC URL.

* kvutils/app: Provide a way to specify the ledger ID.

* ledger-on-sql: If there are missing entries in the log, fail on read.

This can happen right now because we insert in parallel. Next step: stop
doing that.

* ledger-on-sql: Stop writing in parallel; it causes race conditions.

This unfortunately means we also stop _reading_ in parallel, which is
less fun.

* Revert "ledger-on-sql: Use a sequential log entry ID."

This reverts commit c58265bf43.

* ledger-on-sql: Create tables with columns that are not nullable.

* ledger-on-sql: The H2 conformance tests now work, just slowly.

CHANGELOG_BEGIN
CHANGELOG_END

* ledger-on-sql: Run tests against H2 and SQLite on memory and disk.

* ledger-on-sql: Allow H2 to read from the log in parallel with writes.

* ledger-on-sql: Use UUIDs for entry IDs, not 8 random bytes.

* ledger-on-sql: Make sure to log the correlation ID.

* ledger-on-sql: Do less while holding a database connection.

* ledger-on-sql: Log the connection.

* ledger-on-sql: We don't need to tell H2 not to drop the connection.

The connection pool takes care of that.

* ledger-on-sql: Disable H2 conformance tests on CI; they're too slow.

* ledger-on-sql: Rename `loggingContext` to `logCtx`.

* ledger-on-sql: Don't abuse log contexts; put the data in the message.

* ledger-on-sql: Make the connection log line easier to read.

And pull out the logger, even if it's only used once.

* ledger-on-sql: Run the unit tests against (H2, SQLite) * (memory, file).
2020-01-22 16:54:51 +01:00
Stefano Baghino
5231bab423
Introduce ContextualizedLogger and LoggingContext (#4046)
* Introduce ContextualizedLogger and LoggingContext

A ContextualizedLogger is a wrapper around an Slf4j logger. It uses
call-by-name parameters to only construct logging lines when necessary.
The underlying context can be used by invoking withoutContext.

More interestingly, every call to one of its logging methods requires
the presence of an implicit LoggingContext.

A LoggingContext is a way of definining a set of pairs that a
ContextualizedLogger can log without the need of it being specified for
every call.

A new context can be created with newLoggingContext and, given an
implicit LoggingContext, pairs can be added or overwritten using
withEnrichedLoggingContext.

Pairs in the context will be appended to each event logged by a
ContextualizedLogger in the following form:

some text logged explicitly (context: {key1=value1, key2=value2)

If the underlying Logger is configured to use the Logstash encoder the
keys and values in the LoggingContext will also be added to the logging
event encoded in JSON format.

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestion in https://github.com/digital-asset/daml/pull/4046#pullrequestreview-343224692

Thanks to @SamirTalwar

Co-Authored-By: Samir Talwar <samir.talwar@digitalasset.com>

* Address https://github.com/digital-asset/daml/pull/4046#discussion_r366874330

Co-authored-by: Samir Talwar <samir@noodlesandwich.com>
2020-01-16 13:05:40 +01:00
Andreas Herrmann
942c0b7c9e Makes scalatest available in IntelliJ (#4032)
The target override for scalatest was pointing to a meta package that
was empty itself and reexported scalactic and scalatest. It seems this
confused the IntelliJ plugin. This instead introduces dedicated
overrides for scalactic and scalatest.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-13 17:43:52 +00:00
Miklos
1124931f78
Simplified participant-side reader/writer APIs (#3923)
Initial version of simplified participant-side reader/writer APIs.
2020-01-10 12:20:32 +01:00
Brian Healey
cf684d2595
update dropwizrd to version 4.1.2 (#3990)
CHANGELOG_BEGIN
[Ledger API - Metrics] update dropwizard to version 4.1.2
CHANGELOG_END
2020-01-09 07:25:47 -05:00
Brian Healey
9f13a2fb10 Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10 (#3903)
* Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10

Akka 2.6.1 Upgrade Changes
- Materializer in place of ActorMaterializer
- Source.future instead of Source.fromFuture
- The Scheduler.schedule method has been deprecated in favor of selecting scheduleWithFixedDelay or scheduleAtFixedRate
- onDownstreamFinish(cause: Throwable)
- ActorAttributes.supervisionStrategy(...) in place of ActorMaterializerSettings.withSupervisionStrategy

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* Akka 2.6.1 Upgrade Changes
- onDownstreamFinish(cause: Throwable)

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* code review: remove unnecessary supervision strategy
2019-12-19 18:00:59 +00:00
Brian Healey
e8c6364d32
Update akka-http version to 10.1.12, akka-stream version to 2.5.26, and corresponding uptick to scala 2.13 (#3899) 2019-12-18 23:14:28 -05:00
Brian Healey
d8089f44b0
Uptick akka-http version to 10.1.10 / uptick scala version to 2.12.9 (#3896) 2019-12-18 16:22:39 -05:00
Brian Healey
23edf52d36
resolve jackson transitive clash (#3775) 2019-12-09 07:47:53 -05:00
Robert Autenrieth
4c3f6014f6
Implement support for RSA-signed JWT tokens (#3526)
* Improve Navigator output

* Fix Navigator not using the access token

* Add RSA signatures for JWT tokens

* Remove unused method

* Add timeouts for reading JWKS

* Fix test

* Rename method for consistency

* Improve comment

* More renaming for consistency

* CHANGELOG

CHANGELOG_BEGIN
- [Sandbox] Add CLI options to start the sandbox with JWT based authentication with RSA signed tokens.
  See `issue #3155 <https://github.com/digital-asset/daml/issues/3155>`__ .
- [Navigator] Fixed a bug where the `--access-token-file` option did not work correctly.
CHANGELOG_END

* Make JwksVerifier limits configurable

* Make SimpleHttpServer private
2019-11-25 16:29:24 +01:00
mziolekda
0a2641ba78
Remove dead dependency on jboss logging library (#3333)
* remove dead dependency om jboss logging library

* remove references to jboss from license file
2019-11-05 11:36:43 +01:00
Jussi Mäki
4e440bd0a6 Add kvutils ledger dumps and integrity checker (#3167)
kvutils now supports dumping the ledger to a file via environment
variable: "KVUTILS_LEDGER_DUMP=/tmp/ledger.dump".

The integrity checker tool allows re-processing of ledger dumps to
validate that all submissions, state and log entries can be parsed
and processed. This forms the basis of kvutils data continuity test
suite.

Currently the checker strictly enforces that re-processed
outputs match the inputs exactly, which for now guarantees forwards
and backwards compatibility. The checker will be adapted when
forwards-incompatible changes or new kvutils message versions arise.

A select collection of ledger dumps will be maintained in a separate
repository (to be decided) and these dumps are to be validated by
the DAML CI.
2019-10-30 17:02:00 +00:00
Andreas Herrmann
03037ca8bd fetch_sources = True (#3279) 2019-10-29 14:43:56 +00:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00