Commit Graph

72 Commits

Author SHA1 Message Date
akshayshirahatti-da
19fe4266ed
[Trigger Service/Oauth2-Middleware] Hocon config refactor (#12228)
* Changes to add a pureconfig-util module with some shared config readers, and cleanup some code from oauth2-middleware hocon

CHANGELOG_BEGIN
CHANGELOG_END

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/CliSpec.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-01-07 10:35:31 +00:00
akshayshirahatti-da
4f4d18829b
[Trigger-Service] Changes to use a typeconfig conf for trigger-service when provided. (#12217)
* Changes to add the option of starting trigger service with typeconf/HOCON config

CHANGELOG_BEGIN
CHANGELOG_END

* add tests for authorization config and fail on both config file and cli args

* refactor and cleanup config loading and tests

* Changes based on code review comments

* Daml doc changes and making sure that we have defaults for most fields to mirror cli args

CHANGELOG_BEGIN
Trigger Service can now be configured with HOCON config file.
 - If a config file is provided we will choose to start the service using that, else we will fallback to cli arguments.
 - If both config file and cli args are provided we will error out.
CHANGELOG_END

* addressing some more code review comments

* use scalatest inside properly
2022-01-06 00:12:47 +00:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Samir Talwar
9f8e640b1a
Use ExecutionContext.parasitic instead of DirectExecutionContext. (#11783)
* concurrent: Replace `DirectExecutionContextInternal` with `parasitic`.

* concurrent: Rename `DirectExecutionContext` `parasitic`.

* Use `ExecutionContext.parasitic` instead of `DirectExecutionContext`.

We no longer need the latter.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix formatting.
2021-12-10 16:25:46 +01:00
Victor Peter Rouven Müller
99c6be5272
[Trigger-Service] Add support for read as in the trigger service (#11370)
* Add support for read as in the trigger service

changelog_begin

- [Trigger-Service] a list of read_as parties can now be supplied to the playload for the v1/triggers endpoint (i.e. field name is "read_as").

changelog_end

* Fix scala 2.12 build

* Remove not-null constraint from the new read_as column to fix oracle tests

* Fix read queries to respect that the read_as column values can be NULL

* Don't set any default values for the read_as column, it being null is fine

* Add test that covers the readas feature in the trigger service

* Use only alice and public as parties in the new test, don't want to mix up things unnecessarily

* Fix failing auth test

* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Update triggers/service/src/test/scala/com/digitalasset/daml/lf/engine/trigger/TriggerServiceTest.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Address review comments/Add the not null constraint to the read as column for postgres

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-12-10 10:41:13 +01:00
Remy
58e69ade1a
LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
the more consistent, as asked by Moritz in review of #11820

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 12:23:17 +00:00
akshayshirahatti-da
9f8b039be3
[Trigger-Service] Migrate tests to sandbox (#11501)
* Migrate trigger service tests to sandbox from sandbox-classic

CHANGELOG_BEGIN
CHANGELOG_END

* remove config override for EngineMode.Dev based on code review
2021-11-02 14:35:30 +00:00
Robin Krom
7a88c7d97c
trigger-service: dev-mode-unsafe flag (#11233)
* trigger-service: dev-mode-unsafe flag

This adds a `--dev-mode-unsafe` flag analogous to the trigger service
analogous to its counterpart in the trigger runner.

CHANGELOG_BEGIN
CHANGELOG_END

* fix tests
2021-10-13 12:22:10 +00:00
Andreas Herrmann
a1da025b78
FreePort draw from outside ephemeral port range (#10774)
* Test case for LockedFreePort not colliding with port 0

changelog_begin
changelog_end

* Discover dynamic port range on Linux

* Random port generator outside ephemeral range

* remove dev comments

* Draw FreePort from outside the ephemeral port range

Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.

LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.

But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.

* Select dynamic port range based on OS

* Detect dynamic port range on MacOS and Windows

* Import sysctl from Nix on MacOS

changelog_begin
changelog_end

* Windows line separator

* FreePort helpers visibility

* Use more informative exception types

* Use a more light weight unit test

* Add comments

* Fix Windows

* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Add a comment to clarify the generated port range

* fmt

* unused import

* Split libs-scala/ports

Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-09-07 15:41:43 +02:00
Andreas Herrmann
7270ee3c71
Handle dynamic port in auth middleware client (trigger service) (#10755)
* Do not require redirect URI upfront

changelog_begin
changelog_end

This allows users to configure the redirect URI after the middleware
client binding occurred such that dynamic port selection can be taken
into account.

* Use port 0 in trigger service fixture

changelog_begin
changelog_end

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-09-02 18:06:48 +02:00
Samir Talwar
4b8b67a1b5
Upgrade Scalatest to v3.2.9. (#10576)
* Upgrade Scalatest to v3.2.9.

Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.

The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.

As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.

CHANGELOG_BEGIN
CHANGELOG_END

* http-json-oracle: Fix a Scalatest dependency.

* ledger-api-client: Fix a Scalatest dependency.
2021-08-12 23:19:35 +00:00
akshayshirahatti-da
b00e146aa0
[JSON-API/trigger-service] Refactor db conn (#10497)
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils

* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end

* update trigger service to use new libs-scala/db-utils

* missed changes for http-json-oracle

* minor cleanup based on comments

* fix breaking scala 2_12 build

* cleanup db-utils/BAZEL.md file
2021-08-09 09:56:58 +00:00
Samir Talwar
df78f9cb1e
Replace LedgerConfiguration with InitialLedgerConfiguration or the load timeout. [KVL-1058] (#10487)
* participant-integration-api: Encapsulate the initial configuration.

* participant-integration-api: Reduce usage of `LedgerConfiguration`.

* Inline `LedgerConfiguration` wherever it's used.

Most things don't need all its constituent parts; this reduces the
amount of unused properties.

CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
  removed in favor of ``InitialLedgerConfiguration``. Its usage
  has been changed accordingly, with the ``configurationLoadTimeout``
  property becoming part of ``ApiServerConfig`` instead.

  The default options provided by ``LedgerConfiguration`` have been
  removed; you are now encouraged to come up with sensible values for
  your own ledger. The ``Configuration.reasonableInitialConfiguration``
  value may help.
CHANGELOG_END

* Correct the initial configuration submission delay for KV ledgers.

* kvutils: Mark supertype unused parameters as unused.

* kvutils: Extract out common configuration submission delays.

These values are specific to kvutils; other drivers should come up with
their own.

* configuration: Delete `NoGeneration`, as it's unused.
2021-08-05 16:31:45 +00:00
Moritz Kiefer
b7cf42d173
Upgrade doobie to version 0.13.4 (#10326)
Was curious if there were any relevant performance improvements in
newer versions. Looks like the answer is no but we might as well
upgrade anyway.

changelog_begin
changelog_end
2021-07-20 19:19:11 +02:00
Samir Talwar
bd12170af2
participant-state: Remove the ParticipantId, PackageId, and Party aliases. [KVL-1002] (#10308)
* participant-state: Remove the `ParticipantId` alias.

This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Remove the `PackageId` and `Party` aliases.

They don't add anything. Let's just use `Ref`.

* kvutils: Restore missing compat imports.
2021-07-19 12:31:25 +00:00
Samir Talwar
c929a14486
logging-entries: Split from contextualized-logging. [KVL-996] (#10149)
* logging-entries: Split from contextualized-logging.

This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.

CHANGELOG_BEGIN
CHANGELOG_END

* logging-entries: Fix dependencies for 2.12.

* logging-entries: Missed one more Scala 2.12 dependency.

* release: Publish logging-entries.
2021-06-30 12:08:33 +00:00
Moritz Kiefer
0075ff1899
Pass along tags in da_scala_test_suite (#9521)
changelog_begin
changelog_end
2021-04-28 09:14:27 +02:00
Samir Talwar
8cd3658b14
Switch to an environment variable for enabling Oracle tests. (#9511)
* participant-integration-api: Build Oracle tests, but don't run them.

CHANGELOG_BEGIN
CHANGELOG_END

* triggers: Switch to an environment variable for enabling Oracle tests.

* http-json: Switch to an environment variable for enabling Oracle tests.

* Disable running Oracle tests by default, not building them.

* triggers/service: Remove unused test dependencies.
2021-04-27 11:50:39 +00:00
Moritz Kiefer
f0f8027a47
Set supported jdbc driver names at compile time (#9489)
* Set supported jdbc driver names at compile time

This is mainly to unblock the work on Oracle support in the Ledger API
but I think it’s a sensible thing in general. For the Ledger API,
moving the dependency to the top-level is apparently rather
tricky. Because the SDK bundles everything into a single megajar,
Sandbox depending on the oracle library does also result in the JSON
API and the trigger service will also have the oracle library in scope
and will support Oracle in CE which they should not.

This PR simply hardcodes the list of supported drivers to address
that. Not pretty but does the job.

changelog_begin
changelog_end

* format

* Address review comments

changelog_begin
changelog_end
2021-04-26 09:19:31 +02:00
Moritz Kiefer
e84c954973
Expose Oracle support in the EE trigger service (#9342)
* Expose tho Oracle support in the EE trigger service

This PR builds on the previous PR that did all the actual work on
Oracle support and exposes it in the enterprise edition. This PR only
releases the enterprise edition via the SDK tarball. I’ll add
artifactory publishing separately.

changelog_begin
changelog_end

* Update daml-assistant/daml-sdk/validate.sh

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2021-04-08 14:50:47 +02:00
Moritz Kiefer
f7b2f14294
Add Oracle support in the trigger service (#9286)
* Add Oracle support in the trigger service

This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.

changelog_begin
changelog_end

* use getOrElse

changelog_begin
changelog_end
2021-03-31 18:39:08 +02:00
Moritz Kiefer
5d88c08832
Bump ghcide (#9041)
* Bump ghcide

Includes https://github.com/digital-asset/daml-ghcide/pull/13 meaning
we can now remove the hacks for missing signatures

changelog_begin

- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
  produced errors for warnings produced by `-Wmissing-signatures` even
  if the user did not explicitly enable this.

changelog_end

* Bump to merged commit

changelog_begin
changelog_end
2021-03-05 18:17:28 +01:00
Moritz Kiefer
0281b442b8
-Werror all the things (#9027)
We used to do this for some packages but it broke cpp. I don’t
actually know why it doesn’t do that anymore but I’ll gladly accept
that fact and turn it on everywhere.

changelog_begin
changelog_end
2021-03-04 19:08:59 +01:00
Moritz Kiefer
40e1a39f2c
Port //triggers/... to Scala 2.13 (#8781)
changelog_begin
changelog_end
2021-02-09 13:35:05 +01:00
Stefano Baghino
42e071f2c2
Add //libs-scala/doobie-slf4j and replace usages of jdkLogHandler (#8731)
The jdkLogHandler provided by Doobie exists purely as an example and the library
itself does not recommend using it in production.

Note that this slightly changes the runtime behavior, logging successful queries
at debug level rather then info. The message itself is preserved from the original
MIT-licensed example.

This uses Slf4j as most of our components, instead of java.util.logging.

changelog_begin
[HTTP JSON API] The server now logs successful queries at debug level
instead of info
[Trigger Service] The server now logs successful queries at debug level
instead of info
changelog_end
2021-02-03 10:45:05 +00:00
Andreas Herrmann
199c52fd79
Don't always redirect to /login automatically (#8532)
* Don't always redirect to /login automatically

The redirect mode can be configured to never redirect, always redirect,
or redirect based on the request type (redirect for text/html).

In case of no redirect the auth middleware client will reply with 401
Unauthorized with a custom WWW-Authenticate challenge to login on the
auth middleware.

* Make login to redirect configurable on trigger service

By default the trigger service will redirect for HTML requests and not
redirect for JSON requests. The test suite uses automatic redirect as
the OAuth2 test server works without user interaction.

changelog_begin
changelog_end

* Preserve path and query in authMiddlewareUri

This is necessary if the auth middleware lies behind a reverse proxy
with a path prefix or a similar setup.

* Bump default auth middleware login timeout

One minute was to short for a login cycle that requires manual user
input.

* Set token cookie properties

`path = "/"` is required so that the `/login` endpoint can reliably
override the cookie value for other endoints such as `/cb` or
`/v1/triggers`.

* Test redirectToLogin modes

* Redirect on HTML

https://github.com/digital-asset/daml/pull/8532#discussion_r559368335

* Use pass/reject in onRedirectToLogin

8db2bff9af (r559370308)

* default login timeout 5min

https://github.com/digital-asset/daml/pull/8532#discussion_r559535511

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-18 14:03:39 +00:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Andreas Herrmann
4ac7b3f22d
Separate auth API from implementation and testing code (#8183)
* Rename and separate auth targets and packages

Split out the API types `Request`/`Response` into their own Bazel
targets, so that the trigger service does not have to depend on the full
middleware and OAuth2 test server targets.

Also rename packages: Generic auth code goes into `com.daml.auth`,
OAuth2 specific code goes into `com.daml.auth.oauth2` or
`com.daml.auth.middleware.oauth2`.

changelog_begin
changelog_end

* Remove empty test-case

This is a left-over from #7226.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-07 16:17:46 +01:00
Andreas Herrmann
8bceeb13de
Handle token expiry in trigger service (#8037)
* Enable adjustable clock in trigger service tests

changelog_begin
changelog_end

* Test user side token expiry

* Test service side token refresh

* Use AccessToken wrapper in TriggerRunnerImpl

* Store refresh token in trigger DB

* add refresh token to trigger runner config

* TriggerTokenExpired message to server

* TriggerTokenRefresh message to server

* refresh trigger token and update db

* Restart trigger with fresh token

* Test second token expiry

* Refresh token on running trigger

changelog_begin
* [Triggers] UNAUTHENTICATED errors will now terminate the trigger.
  These errors are no longer available for handling in the trigger DAML
  code. Instead, they are forwarded to the trigger service for handling,
  e.g. access token refresh.
changelog_end

* todo note

* Move triggerRunnerName and getRunner into object

* Factor out token refresh

* Factor out getActiveContracts

* factor out create command

* Add logging to token refresh

* Handle token expiry in TriggerRunner

TriggerRunnerImpl throws a dedicated exception when it fails on an
expired access token (any unauthenticated error to be precise).
The TriggerRunner supervisor reacts to this child failure by
requesting a token refresh and restart on the trigger server and
stopping itself.
The trigger server requests a new access and refresh token on the auth
middleware and restarts the trigger.

This works around an issue with actor supervisors in akka-actor-typed.
A stop supervisor wrapped within a restart supervisor will not cause a
stop as expected. Instead, the restart supervisor will trigger as well
and restart the actor. The work around uses a custom behavior
interceptor to emulate the appropriate stop supervisors as closely as
possible. We cannot properly emulate ChildFailed signals this way, so
we use dedicated messages intead.

* throw --> Future.failedo

* getOrFail helper

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-02 16:17:45 +00:00
Samir Talwar
9976b4cd50
Bazel: Factor out logic around Protobuf JARs. [KVL-714] (#8084)
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.

* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.

* Bazel: Factor out Java/Scala protobuf class generation into a helper.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/archive: Use `proto_jars`.

* Bazel: Remove the visibility modifier from `proto_jars`.

It's too confusing. Just make everything public.

* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.

* Bazel: Add comments to the various parts of `proto_jars`.

* daml-assistant: Do unpleasant things with `location` in Bazel.
2020-11-27 08:34:53 +00:00
Moritz Kiefer
21aaf8b173
Use futures for database interaction in trigger service (#7999)
akka-http gets a bit unhappy if you block within requests and we also
use the unsafeToFuture in the JSON API so it seems sensible to do the
same in both.

I’ve moved out the initDb option out of the Server actor both because
it seemed cleaner than calling sys.exit in the actor and because it
was annoying to fit it in.

changelog_begin
changelog_end
2020-11-19 09:13:05 +01:00
Andreas Herrmann
14f71ed20c
Split trigger service tests into a test suite (#7983)
This is to reduce the likelyhood of these tests timint out. Bazel will
generate a dedicated test target per `.scala` file. Meaning the tests
can run in parallel and each test target should have a shorter overall
runtime.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-17 13:14:24 +00:00
Moritz Kiefer
dbf28e0794
Revamp trigger status endpoint (#7961)
* Revamp trigger status endpoint

fixes #7951

The previous endpoint was a memory leak, nothing got persisted across
restarts and it omitted useful information like the metadata of the
trigger. The information is useful for testing, so I abstracted over
it so we can do what we did before in testing.

As for the endpoint, it now queries the actor for its current status
and only returns that and includes the metadata in the response.

As mentioned in #7951, I do think there is value in some kind of
history and potentially something including trace statements but I’d
like to do that properly instead of the hacky thing we have atm.

changelog_begin

- [Trigger Service] The trigger status endpoint /v1/triggers/:id now
  includes metadata about the trigger like the party and the trigger
  id. The logs field has been replaced by a status field.

changelog_end

* Fix ACS query test

changelog_begin
changelog_end
2020-11-13 15:59:34 +01:00
Andreas Herrmann
9758b2f85b
Check trigger dao migrations digests (#7908)
* Check the trigger dao migrations digest

Following the example of the corresponding ledger on SQL tests.

The digests had to be updated as both of them had gone out of sync.
The init digest presumably due to the change in #7226 and the one for
adding the access token during review of #7890.

changelog_begin
changelog_end

* define abstract migrations test

* Use abstract migrations test in trigger service tests

* use abstract migrations test in ledger on SQL

* Retain check for number of .sql resources

* Factor out the hash-migrations script

* Consistent shell settings

Addressing review comment

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-06 09:20:32 +00:00
Moritz Kiefer
31a3f25800
Add a --port-file option to the trigger service (#7907)
fixes #7097

changelog_begin

- [Triggers] The trigger service now has a `--port-file` option
  matching the corresponding option in the JSON API.

changelog_end
2020-11-05 15:03:34 +01:00
Andreas Herrmann
dc7ec9ef45
Refactor trigger service test fixture (#7880)
* Separate trigger service test fixture

Define a dedicated fixture per resource instead of generating all
resources in a single fixture function. This allows to control the
lifetime of resources individually.

Reuse the same sandbox and toxiproxy and auth middleware for the test
suite instead of restarting new resources for each test case. This is to
reduce overall test runtime.

changelog_begin
changelog_end

* fix exec context

changelog_begin
changelog_end

* Use owned resources where appropriate

Avoid Await result and use owned resources instead.

* Reduce test timeout

* Be more patient for Windows

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-11-04 15:00:28 +00:00
Andreas Herrmann
cd56af1cdd
Factor out the JWT verifier CLI options (#7780)
* Factor JWT verifier CLI flags

changelog_begin
changelog_end

* Use cli-opts in auth middleware

* Use cli-opts in sandbox cli

* Mark trigger service test as long

These have become prone to timeout on CI.

Increasing the size (timeout) is a temporary fix. A proper
solution is to a) not start a fresh sandbox per test-case and b)
separate the in-mem/db and no-auth/auth configrations into
separate Bazel test targets.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-10-26 15:10:40 +01:00
Andreas Herrmann
60fe244e1b
Use auth middleware in trigger service /v1/start endpoint (#7654)
* Authorize trigger service on middleware

changelog_begin
changelog_end

* Trigger service auth callback handler

* Forward token

* Do not pin the application ID in the access token

The trigger service will assign an individual application ID to each
trigger based on its UUID. Requiring tokens on the granularity of
application IDs would break the idea of storing the token in a cookie to
be able to use it across multiple requests.

changelog_begin
changelog_end

* todo persist trigger token

* Add a state parameter to middleware login

* add documentation comments

* typo

* fmt

* Align Party type between middleware and trigger service

The middleware was using `com.daml.lf.data.Ref.Party` while the trigger
service is using `com.daml.ledger.api.refinements.ApiTypes.Party` which
requires conversions. This aligns the types to avoid such conversions.

* optional application id in oauth2 test server

* align party types

* configure auth middleware in trigger service tests

* handle empty cookie header

* follow redirects in trigger service tests

* keep track of cookies

* keep track of cookies

* Replace any previous Cookie header

Otherwise on old daml-ledger-token cookie might persist and be preferred
over a newly added instance.

* DEBUG

* Configure test ledger client readAs claims

* fmt

* docstrings

* remove debug output

* Avoid endless redirect loops

When the replay still fails to authorize on the middleware then we do
not want to attempt another login flow.

* Store callback routes in authCallbacks

* fmt

* Push AuthTestConfig into test target

https://github.com/digital-asset/daml/pull/7654#discussion_r506510193

* Unbind oauth2 server after middleware

https://github.com/digital-asset/daml/pull/7654/files#r506513251

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-10-16 15:37:36 +00:00
Remy
a899e5e4e6
Speedy: Check language version as part of Speedy compilation. (#7440)
Additionnally
+ move allowedLanguageVersion inside compiler config
+ add missing catch error insde
  ConcurrentCompiledPackages#addPackageInternal
+ implement EngineConfig method to easily produce a Compiler.Config

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-22 12:43:48 +02:00
Moritz Kiefer
3171bb588a
Migrate trigger assertion library to DAML Script (#7318)
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).

changelog_begin

- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.

changelog_end
2020-09-04 13:48:50 +02:00
Stephen Compall
0b72b6f716
more trigger service logs; tagged LoggingContext (#7205)
* get a LoggingContext into the TriggerRunnerImpl

* make some implicits more implicitly scoped and explicitly ascribed

* make some private/final markings

* most of JsonFormat[Identifier] is in companion

* experimental LoggingContext with phantom type

* ActorContext#log isn't really doing that much

* more details of LoggingContextOf

* make LoggingContextOf compile

* add trigger message logging, yet without context

* fix parent compile errors

* use Config as the phantom for its own logging extensions

* LocalDateTimeFormat cleanup

* switch TriggerRunner to contextual logging

* add trigger definition ID to logs

* log trigger-submitted commands, fix trigger test compile

* log trigger stopping and DAR uploads

* add context to PostStop/PreRestart logs

* add changelog

CHANGELOG_BEGIN
- [Triggers] More detailed logging of trigger actions and trigger service actions.
  See `issue #7205 <https://github.com/digital-asset/daml/pull/7205>`_.
CHANGELOG_END

* missed copyright header

* switch to Unit, scala/bug#9240 fixed
2020-09-03 17:47:09 +00:00
Samir Talwar
2b3cf1b3c0
ledger-api-client: Rename maxInboundMessageSize to maxInboundMetadataSize. (#7290)
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.

CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
  to ``maxInboundMetadataSize``, to match the builder Netty channel
  builder. It was incorrectly named ``maxInboundMessageSize``, which is
  a different channel property that configures the maximum message size,
  not the header size.
CHANGELOG_END

* ledger-api-client: Introduce a `maxInboundMessageSize` config property.

We use this a lot; easier if it's in the configuration.

CHANGELOG_BEGIN
- [Scala Bindings] Replace the
  ``LedgerClientConfiguration.maxInboundMessageSize`` property with a
  new one that represents the maximum size of the response body.
CHANGELOG_END
2020-09-02 08:41:34 +00:00
Moritz Kiefer
6e4f101aad
Remove ref-ledger-authenticator from trigger service (#7226)
* Remove authentication service from trigger service

This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).

While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.

changelog_begin

- [Trigger Service] Parties are now specified in request bodies as
  opposed to via HTTP Basic auth. This is done in preparation for
  running the trigger service against authenticated ledgers.

changelog_end

* Fix integration tests

changelog_begin
changelog_end
2020-08-27 15:36:59 +02:00
Stephen Compall
2369a3bbae
use sandbox-style flyway migrations for trigger service (#7092)
* depend on flyway in trigger service; duplicate FlywayMigrations from sandbox

* bring in postgres/h2 detection, other data-source elements needed; use scalalogging

* use DataSource instead of DriverManager for DbTriggerDao, pass to flyway too

- initialization order isn't quite right yet

* hardcode postgres in triggers flyway path

* found an init order that doesn't fail in tests

* better pools for production, and don't forget to close them

* move most init SQL to first migration file

* make logHandler implicit again

- seems to have been an accidental implication of #6334

* move remaining create to V1__Init.sql

* use Prod poolSize

* prove that PostStop signals are never seen (in tests)

* move all the shutdown to Stop message

* remove dead code in DbFlywayMigrations

* always "initialize" database

CHANGELOG_BEGIN
- [Trigger Service] The database format has changed to allow migrations in
  future releases.  Databases are always initialized or migrated to the current
  version on start, so use of ``--init-db`` is no longer required.
  See `issue #7092 <https://github.com/digital-asset/daml/pull/7092>`__.
CHANGELOG_END

* it should always be obvious when there is no escape

* remove logs from experiments

* remove receiveSignal PostStop

* remove unused imports
2020-08-20 18:52:37 +00:00
Remy
e54c182032
Engine: move check of valide packages inside MutableCompiledPackages (#7186)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-19 19:37:53 +02:00
Stephen Compall
1737907415
add --address option to trigger service (#7090)
* factor --address, --http-port, --port-file options from http-json to cli-opts

- enabling reuse in trigger service

* use cli-opts for address and http-port options in Trigger service

* mark ServiceConfig and some defaults private

* use --address option to set up server

* document Setter

* test --address option is parsed

* missing (c) headers

* add changelog

CHANGELOG_BEGIN
- [Trigger Service] Accepts a new ``--address`` option to listen for HTTP connections on
  interfaces other than localhost, such as ``0.0.0.0`` for all addresses.
  See `issue #7090 <https://github.com/digital-asset/daml/pull/7090>`__.
CHANGELOG_END
2020-08-12 13:50:00 -04:00
Stephen Compall
0857c28874
turn on NonUnitStatements in trigger runner, service main code (#7079)
* turn on NonUnitStatements in trigger runner, service main code

* add some friendly discards

* add a not-so-friendly discard

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* one more discard to clarify an Await.ready's type

* explain unbind issue
2020-08-11 14:30:11 +00:00
Rohan Jacob-Rao
91b174da91
Trigger service tests: Remove auth service process in trigger service fixture (#6928)
This code has a few problems: the auth service test mode is not enabled, the
admin ledger should be separate from main ledger if auth is enabled, and the
retry strategy not executed.

The auth service is not used in these tests yet and we have an example
of how to do it correctly in the auth service fixture (and auth service
client tests). We can try to reuse that fixture instead or copy the
code over when necessary.

changelog_begin
changelog_end
2020-07-29 23:29:04 +00:00
Rohan Jacob-Rao
4973bd3573
Trigger service: rest of auth service client (#6804)
* Method for service account request

* Extend domain and factor out sa/secure

* Methods to list service accounts and poll for service account

changelog_begin
changelog_end

* Run auth client tests 20 times

* Revert "Run auth client tests 20 times"

This reverts commit 5264ce6f30.

* Assert creds list for new service account is empty

* Request credential method

* Method to get a new credential id

* Method for fetching credential

* Login method (for ledger access token)

* Make authorize style consistent with rest

* Remove redundant test (subset of second test)

* Run auth client tests 20 times

* Fail future for http error responses and refine retry strategy

* Scalafmt

* Revert "Run auth client tests 20 times"

This reverts commit 2568dc6b47.

* Test getting service account before request
2020-07-21 21:26:49 +00:00
Rohan Jacob-Rao
9e979ced9a
Trigger service: Introduce auth service client and test fixture (#6768)
* WIP Call authorize before start trigger

* Intro auth service client

* Auth service client test setup

CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt

* Use Spray JSON format and slightly better uri handling

* Get auth client test to work and fiddle with uris

* Remove changes to Server

* Add package for URI

* Format

* Copyrights

* Update sandbox packages

* Use default wallclock time for auth service sandbox
2020-07-20 13:35:14 +00:00