Commit Graph

323 Commits

Author SHA1 Message Date
Moritz Kiefer
40e1a39f2c
Port //triggers/... to Scala 2.13 (#8781)
changelog_begin
changelog_end
2021-02-09 13:35:05 +01:00
Stefano Baghino
42e071f2c2
Add //libs-scala/doobie-slf4j and replace usages of jdkLogHandler (#8731)
The jdkLogHandler provided by Doobie exists purely as an example and the library
itself does not recommend using it in production.

Note that this slightly changes the runtime behavior, logging successful queries
at debug level rather then info. The message itself is preserved from the original
MIT-licensed example.

This uses Slf4j as most of our components, instead of java.util.logging.

changelog_begin
[HTTP JSON API] The server now logs successful queries at debug level
instead of info
[Trigger Service] The server now logs successful queries at debug level
instead of info
changelog_end
2021-02-03 10:45:05 +00:00
Andreas Herrmann
a226324afa
Use cliopts.Http for OAuth 2.0 middleware (#8671)
* Use cliopts.Http for OAuth 2.0 middleware

changelog_begin
- [OAuth 2.0 Middleware] You can now configure the address that the
  middleware listens to using the ``--address`` flag.
  The port that the middleware listens to is now configured using the
  ``--http-port`` flag, use 0 to dynamically choose a free port.
  You can now configure a port file where the chosen port will be
  written to using the ``--port-file`` flag.
changelog_end

* Add test-case for OAuth 2.0 middleware port file

* Don't forget to close source

* Fix integration test

* Update triggers/service/auth/src/main/scala/com/daml/auth/middleware/oauth2/Config.scala

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-01-29 09:32:07 +00:00
Remy
de14275528
triggers: do not use the type information from SRecord (#8623)
This type information is there for debugging purpose, one should not
use it.  We use type from SAny instead.

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-26 12:08:53 +01:00
Andreas Herrmann
5706f5e15d
Drop early access tag on trigger service and auth middleware (#8611)
* add livez endpoint to auth middleware

* Add OAuth 2.0 middleware to Daml SDK

* unhide trigger service auth flags

changelog_begin
- [Triggers] The trigger service now supports authorization through an
  auth middleware. The feature is enabled using the `--auth` and
  `--auth-callback` command-line flags. Please refer to the
  Authorization chapter of the trigger service documentation for further
  instructions.
- [OAuth 2.0 middleware] Daml Connect now includes an implementation of
  the auth middleware API that supports OAuth 2.0 Authorization Code
  Grant. Please refer to the Auth Middleware and OAuth 2.0 Auth
  Middleware chapters of the documentation.
changelog_end

* drop early access flag on triggers

Daml triggers, the trigger service, and the auth middleware are no
longer marked as early access features.

changelog_begin
- [Triggers] Daml Triggers and the Trigger Service are no longer in
  early access status.
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-22 17:02:24 +01:00
Gary Verhaegen
ce8c38bac1
copyright updates (#8610)
CHANGELOG_BEGIN
CHANGELOG_END
2021-01-22 15:03:36 +00:00
Andreas Herrmann
7543c6fdc0
Document authorization with trigger service (#8581)
* Docs for trigger-service auth and auth middleware

changelog_begin
changelog_end

* Document OAuth 2.0 authorization middleware

* Explain 401 Unauthorized resonse

* Add JS frontend example

* Update docs/source/tools/auth-middleware/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/tools/auth-middleware/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/tools/auth-middleware/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/tools/auth-middleware/index.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/tools/auth-middleware/oauth2.rst

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* trigger service don't redirect by default

3a5dba7a70 (r562497928)

* Call it Auth Middleware

https://github.com/digital-asset/daml/pull/8581#discussion_r562442571

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-01-22 11:53:26 +01:00
Andreas Herrmann
33e13aeb3d
Improve browser compatbility of auth middleware (#8590)
* Include auth challenge in response body

changelog_begin
changelog_end

Some browsers make it difficult to access the `WWW-Authenticate`
response header from Javascript. For example, Firefox 84.0.2 (64-bit) on
Linux does not expose the `WWW-Authenticate` header in the result of the
`fetch` function, independent of the server's access control headers.
In that case the header is only accessible through the `XMLHttpRequest`
API, which is more cumbersome to use.

This adds the challenge to the response body in JSON format as well to
avoid exposing users to any such browser related issues.

* Make Secure Set-Cookie attribute configurable

changelog_begin
changelog_end

Since Chrome 80 Set-Cookie with the Secure attribute enabled is rejected
for connections that don't use https [1]. This includes localhost.
Firefox, at least as of version 84.0.2, allows such cookies on
localhost.

This adds a command-line flag to the authorization middleware to make
the value of the Secure attribute configurable. This way it can be
disabled for development purposes.

[1]: https://blog.chromium.org/2019/10/developers-get-ready-for-new.html

* Expand description on --cookie-secure flag

* Fix TriggerServiceFixture

* Fix middleware TestFixture

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-21 16:10:09 +00:00
Andreas Herrmann
199c52fd79
Don't always redirect to /login automatically (#8532)
* Don't always redirect to /login automatically

The redirect mode can be configured to never redirect, always redirect,
or redirect based on the request type (redirect for text/html).

In case of no redirect the auth middleware client will reply with 401
Unauthorized with a custom WWW-Authenticate challenge to login on the
auth middleware.

* Make login to redirect configurable on trigger service

By default the trigger service will redirect for HTML requests and not
redirect for JSON requests. The test suite uses automatic redirect as
the OAuth2 test server works without user interaction.

changelog_begin
changelog_end

* Preserve path and query in authMiddlewareUri

This is necessary if the auth middleware lies behind a reverse proxy
with a path prefix or a similar setup.

* Bump default auth middleware login timeout

One minute was to short for a login cycle that requires manual user
input.

* Set token cookie properties

`path = "/"` is required so that the `/login` endpoint can reliably
override the cookie value for other endoints such as `/cb` or
`/v1/triggers`.

* Test redirectToLogin modes

* Redirect on HTML

https://github.com/digital-asset/daml/pull/8532#discussion_r559368335

* Use pass/reject in onRedirectToLogin

8db2bff9af (r559370308)

* default login timeout 5min

https://github.com/digital-asset/daml/pull/8532#discussion_r559535511

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-18 14:03:39 +00:00
Moritz Kiefer
427ea0fb7d
Remove redundant list of LF versions (#8490)
* Remove redundant list of LF versions

After #8472, I realized that there must be a list used for daml-stdlib
and daml-prim already and it turns out there is. I’ve removed that one
in favor of the one added in #8472 since I like having all in one
place and the one from #8472 is created by filtering an existing list
instead of creating a completely separate list like we do here.

changelog_begin
changelog_end

* Introduce SCRIPT_LF_VERSIONS

changelog_begin
changelog_end
2021-01-13 13:21:39 +00:00
Andreas Herrmann
7855b023ac
Oauth template (#8453)
* Use ActorSystem and ExecutionContext from RequestContext

* Factor out middleware server class

To avoid passing around config and state manually.

changelog_begin
changelog_end

* Depend on databricks/sjsonnet

changelog_begin
changelog_end

* Generate request params from jsonnet template

changelog_begin
changelog_end

* Split middleware test suite sources

* Add test suite for request templates

* fmt

* TriggerServiceFixture template arguments

* Use null to indicate missing applicationId claim

Addressing
https://github.com/digital-asset/daml/pull/8453/files#r555025173

* Fix invalid path on Windows

* Close request template source

* Avoid repeated re-reading of Jsonnet files

https://github.com/digital-asset/daml/pull/8453/files#r555044262

* Factor out template argument mappings

* factor out template error handling

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-11 17:27:46 +00:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Bernhard Elsner
cda93db944
Daml case and logo (#8433)
* Replace many occurrences of DAML with Daml

* Update docs logo

* A few more CLI occurrences

CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END

* Fix some over-eager replacements

* A few mor occurrences in md files

* Address comments in *.proto files

* Change case in comments and strings in .ts files

* Revert changes to frozen proto files

* Also revert LF 1.11

* Update get-daml.sh

* Update windows installer

* Include .py files

* Include comments in .daml files

* More instances in the assistant CLI

* some more help texts
2021-01-08 12:50:15 +00:00
Andreas Herrmann
ef26c08ef8
Make auth callback state bounded (#8351)
* Bounded auth middleware client callback store

changelog_begin
changelog_end

* Json format for login response

* Add middleware client binding to test fixture

* Test middleware client callback store size

* Make max auth callbacks and timeout configurable on trigger service

changelog_begin
changelog_end

* Bounded pending login requests at auth middleware

* Make max logins and timeout configurable on middleware

* Test middleware login store size

* fmt

* Fix Windows

failed with `"localhost" != "127.0.0.1"`. Hardcode "localhost" to avoid
platform specific resolution.

* Use FiniteDuration for login timeout

* Respond with 503 on full login request store

Addresses https://github.com/digital-asset/daml/pull/8351#pullrequestreview-560859604

* Add tests for RequestStore

* Lower DefaultMaxLoginRequests

b48050eb91 (r552649275)

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-01-06 16:02:05 +00:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Remy
9ba146a3e4
LF: Preview of LF 1.11 (#8280)
First step of LF 1.11 release #7139

Preview version is not stable and cannot be used in the ledger only in dev mode.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-18 17:08:22 +01:00
Andreas Herrmann
20cf6fcbd0
Make auth callback URIs configurable (#8342)
* Make middleware callback URI configurable

changelog_begin
changelog_end

* Make trigger service callback URI configurable

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-18 14:39:27 +00:00
Andreas Herrmann
652c9428ed
Use execution context, actor system, and materialzer from request context in route (#8300)
* Extract actor system and execution context from request context

* Extract actor system and execution context from request context

* Extract actor system, materializer, and execution context from request context

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-16 09:41:42 +01:00
Sofia Faro
e52c2a0252
Bundle ghcversion.h and hpp with damlc. (#8278)
* Bundle hpp with damlc.

This PR eliminates the red squiggly lines you get when opening up the
standard library in daml studio (e.g. after a Go To Definitien).

(When CPP is used, Go To Definition is still pointing to the wrong place,
because the line numbers are messed up. But it's better than having a
million error messages.)

We could also remove --cpp flag in a separate PR, though that may be
considered a breaking change.

changelog_begin
changelog_end

* cleanup locateRunfiles jank

* Revert "cleanup locateRunfiles jank"

This reverts commit 82552003ae.
2020-12-14 17:23:41 +00:00
Gary Verhaegen
73724bccb8
protect trigger migrations (#8265)
This prevents the `dade-copyright-headers` script from changing those
files (and from checking them for copyright headers, unfortunately).

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-11 17:05:37 +01:00
Andreas Herrmann
d3b4042ade
Factor out the middleware client from the trigger server (#8244)
* Factor out authorization middleware client

changelog_begin
changelog_end

* Factor out token refresh

* Factor out auth request

* Factor out auth middleware URIs

* factor out auth directive

* Factor out login directive

* Add login response type to api library

* Allow error handling in login callback

* Factor error handling out of authorize directive

* Move tagged token types into middleware api

* Factor out the auth middleware client

* Dedicated exception types in middleware client

* Handle auth middleware client exception

Restores the behavior of authorize before factoring out the client

* expose middleware URIs

* Use the middleware client in the middleware tests

* Use localhost/CALLBACK in testing

https://github.com/digital-asset/daml/pull/8244#discussion_r540328001

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-10 17:26:06 +00:00
Andreas Herrmann
bd09e8265d
Require authorization on DAR upload endpoint (#8193)
* Test authentication on upload_dar endpoint

changelog_begin
changelog_end

* require authentication on upload_dar endpoint

* push Directive into auth

* Fully upload request before auth redirection

* Make HTTP entity upload parameters configurable

changelog_begin
changelog_end

* Shorten help message

https://github.com/digital-asset/daml/pull/8193#discussion_r538428368

* maxHttpEntityUploadSize as Long

https://github.com/digital-asset/daml/pull/8193#discussion_r538431773

* use DefaultMaxInboundMessageSize for DefaultMaxHttpEntityUploadSize

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 18:09:06 +01:00
Andreas Herrmann
59dae3c97b
Handle admin claims in OAuth2 test server (#8185)
* Middleware test resources

- Make OAuthServer accessible
- Don't access suiteResource directly in the test-suite

* OAuth2 test server test resources

- Make Server accessible
- Don't access suiteResource directly in the test-suite

* Use a deny list in OAuth2 test server

It is simpler to configure a set of disallowed parties and extend it on
demand than switching between blanket approval and allow list modes.

changelog_begin
changelog_end

* Handle admin claims in test server

Requests with admin claims are granted by default and denied if admin
requests have been disallowed.

* reset admin state in test fixtures

* test admin tokens in OAuth2 test server

* test admin claims in OAuth2 middleware

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 10:16:29 +00:00
Andreas Herrmann
e1b1e4d646
Wait for trigger shutdown before restart (#8186)
changelog_begin
changelog_end

This is to avoid a race condition where the old trigger runner may not
yet have completed shutdown by the time that the token has been
refreshed and we attempt to start a new trigger runner.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 09:43:51 +00:00
Moritz Kiefer
7e05dc7932
Upgrade rules-scala and scalatest (#8187)
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.

This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.

Apologies for the giant PR, I don’t see a way to keep it smaller.

changelog_begin
changelog_end
2020-12-08 06:59:23 +01:00
Andreas Herrmann
4ac7b3f22d
Separate auth API from implementation and testing code (#8183)
* Rename and separate auth targets and packages

Split out the API types `Request`/`Response` into their own Bazel
targets, so that the trigger service does not have to depend on the full
middleware and OAuth2 test server targets.

Also rename packages: Generic auth code goes into `com.daml.auth`,
OAuth2 specific code goes into `com.daml.auth.oauth2` or
`com.daml.auth.middleware.oauth2`.

changelog_begin
changelog_end

* Remove empty test-case

This is a left-over from #7226.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-07 16:17:46 +01:00
Andreas Herrmann
8bceeb13de
Handle token expiry in trigger service (#8037)
* Enable adjustable clock in trigger service tests

changelog_begin
changelog_end

* Test user side token expiry

* Test service side token refresh

* Use AccessToken wrapper in TriggerRunnerImpl

* Store refresh token in trigger DB

* add refresh token to trigger runner config

* TriggerTokenExpired message to server

* TriggerTokenRefresh message to server

* refresh trigger token and update db

* Restart trigger with fresh token

* Test second token expiry

* Refresh token on running trigger

changelog_begin
* [Triggers] UNAUTHENTICATED errors will now terminate the trigger.
  These errors are no longer available for handling in the trigger DAML
  code. Instead, they are forwarded to the trigger service for handling,
  e.g. access token refresh.
changelog_end

* todo note

* Move triggerRunnerName and getRunner into object

* Factor out token refresh

* Factor out getActiveContracts

* factor out create command

* Add logging to token refresh

* Handle token expiry in TriggerRunner

TriggerRunnerImpl throws a dedicated exception when it fails on an
expired access token (any unauthenticated error to be precise).
The TriggerRunner supervisor reacts to this child failure by
requesting a token refresh and restart on the trigger server and
stopping itself.
The trigger server requests a new access and refresh token on the auth
middleware and restarts the trigger.

This works around an issue with actor supervisors in akka-actor-typed.
A stop supervisor wrapped within a restart supervisor will not cause a
stop as expected. Instead, the restart supervisor will trigger as well
and restart the actor. The work around uses a custom behavior
interceptor to emulate the appropriate stop supervisors as closely as
possible. We cannot properly emulate ChildFailed signals this way, so
we use dedicated messages intead.

* throw --> Future.failedo

* getOrFail helper

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-02 16:17:45 +00:00
Samir Talwar
9976b4cd50
Bazel: Factor out logic around Protobuf JARs. [KVL-714] (#8084)
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.

* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.

* Bazel: Factor out Java/Scala protobuf class generation into a helper.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-lf/archive: Use `proto_jars`.

* Bazel: Remove the visibility modifier from `proto_jars`.

It's too confusing. Just make everything public.

* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.

* Bazel: Add comments to the various parts of `proto_jars`.

* daml-assistant: Do unpleasant things with `location` in Bazel.
2020-11-27 08:34:53 +00:00
Moritz Kiefer
2d3820ac14
Upgrade akka-http to 10.2 (#8058)
* Upgrade akka-http to 10.2

Follow up to #8048, I left out this upgrade to reduce noise and since
I wasn’t quite sure how involved it was going to be.

changelog_begin
changelog_end

* Reenable transparent HEAD requests

Apparently no longer on by default but we depend on this in waitForHttpServer

changelog_begin
changelog_end
2020-11-25 13:39:25 +01:00
Moritz Kiefer
d83cbdb475
Upgrade akka and akka-http (#8048)
* Upgrade akka and akka-http

Was chasing an issue somewhere and thought this might affect it in
some way. It didn’t but I might as well turn the upgrade into a PR.

changelog_begin
changelog_end

* Fix trigger service tests

changelog_begin
changelog_end

* Downgrade akka-http again

changelog_begin
changelog_end

* Upgrade akka-http again and fix tests

changelog_begin
changelog_end

* Cleanup trigger service

changelog_begin
changelog_end
2020-11-25 10:13:51 +01:00
Moritz Kiefer
9a6c33a7e3
Pass application_id through auth middleware (#8011)
fixes #7978

There is no new test in the trigger service since the existing test
for the custom application id already hits this. The difference is
that now the test authorization server will produce a token with the
application id set to what we request rather than the wildcard token
we used before.

changelog_begin
changelog_end
2020-11-19 12:00:54 +01:00
Moritz Kiefer
21aaf8b173
Use futures for database interaction in trigger service (#7999)
akka-http gets a bit unhappy if you block within requests and we also
use the unsafeToFuture in the JSON API so it seems sensible to do the
same in both.

I’ve moved out the initDb option out of the Server actor both because
it seemed cleaner than calling sys.exit in the actor and because it
was annoying to fit it in.

changelog_begin
changelog_end
2020-11-19 09:13:05 +01:00
Andreas Herrmann
59f40cb54e
Hide docs of Daml.Trigger.Internal (#7992)
changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-18 12:26:06 +00:00
Andreas Herrmann
14f71ed20c
Split trigger service tests into a test suite (#7983)
This is to reduce the likelyhood of these tests timint out. Bazel will
generate a dedicated test target per `.scala` file. Meaning the tests
can run in parallel and each test target should have a shorter overall
runtime.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-17 13:14:24 +00:00
Andreas Herrmann
90dc3a5669
Implement token refresh in auth middleware (#7981)
* Obtain refresh token from Auth0

Auth0 requires the `offline_access` scope to be set to return a refresh
token.

See https://auth0.com/docs/tokens/refresh-tokens/get-refresh-tokens

Additionally, the `audience` claim needs to be set to obtain a JWT
access token and a refresh token.

See https://auth0.com/docs/tokens/refresh-tokens

changelog_begin
changelog_end

* Implement refresh endpoint on auth middleware

Following the refresh spec [1] and Auth0 documentation [2].

[1]: https://tools.ietf.org/html/rfc6749#section-6
[2]: https://auth0.com/docs/tokens/refresh-tokens/use-refresh-tokens

* Adapt Auth0 example configuration

Ignore any requests outside the ledger-api audience.

Don't throw on missing query fields. Otherwise the unhandled exception
would prevent unrelated requests from succeeding. E.g. token refresh
requests would always fail.

* Forward unauthorized/forbidden response on refresh

* re-use precomputed token payload

* Implement token refresh in auth test server

Reuses the association between authorization code and token payload to
associate refresh tokens and token payload.

Adds an expiry to the generated token to make tokens distinguishable
across refresh.

* obtain refresh token in test client

* Test auth server refresh token

* auth test server clock configurable

The clock used to define token expiry is configurable

* Override default clock in test fixture

* implement an adjustable clock

* Test token refresh with adjustable clock

* Test token expiry on /auth backend

* Test case for auth middleware /refresh endpoint

* handle malformed code/refresh token in auth server

* Forward client errors on middleware refresh

* Test middleware refresh failure

* Clarify meaning of offline accesss

* Remove redundant testing only comment

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-17 12:06:42 +00:00
Moritz Kiefer
10792ed6fd
Make application ID configurable in trigger service (#7974)
* Make application ID configurable in trigger service

fixes #7671

changelog_begin

- [Trigger Service] The application id used by a trigger can now be
  configured by an optional `applicationId` in the start request.

changelog_end

* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/Server.scala

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-11-17 10:25:30 +01:00
Moritz Kiefer
d8bbd2e8bd
Cleanup trigger message types (#7964)
* Cleanup trigger message types

This properly separates the messages accepted by the internal
TriggerRunnerImpl from the ones accepted by TriggerRunner. This also
shows a bug/redundancy where we had a bunch of code in
TriggerRunnerImpl to handle `Stop` but as evidenced by the types now,
we never actually send this message. We send it to TriggerRunner which
then tears down the child with it.

It also shuffles around the server message type to make it clear where
it belongs to.

And of course, I managed once again to include debugging output from a
previous PR so this is now removed as well 🤦

changelog_begin
changelog_end

* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/TriggerRunnerImpl.scala

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2020-11-16 12:27:51 +01:00
Moritz Kiefer
b6e04c4fee
Fix protobuf decoding in trigger service (#7967)
We need to go via the methods in Reader to make sure that we get our
increased protobuf recursion limit. Otherwise, we fail when trying to
read from the database on anything non-trivial. I’ve verified that the
definition I’ve added is sufficient to break the default limit.

changelog_begin

- [Trigger Service] Fix a bug where complex models resulted in a fatal
  error when restoring the state from the database due to an incorrect
  protobuf recursion limit.

changelog_end
2020-11-16 12:16:03 +01:00
Moritz Kiefer
dbf28e0794
Revamp trigger status endpoint (#7961)
* Revamp trigger status endpoint

fixes #7951

The previous endpoint was a memory leak, nothing got persisted across
restarts and it omitted useful information like the metadata of the
trigger. The information is useful for testing, so I abstracted over
it so we can do what we did before in testing.

As for the endpoint, it now queries the actor for its current status
and only returns that and includes the metadata in the response.

As mentioned in #7951, I do think there is value in some kind of
history and potentially something including trace statements but I’d
like to do that properly instead of the hacky thing we have atm.

changelog_begin

- [Trigger Service] The trigger status endpoint /v1/triggers/:id now
  includes metadata about the trigger like the party and the trigger
  id. The logs field has been replaced by a status field.

changelog_end

* Fix ACS query test

changelog_begin
changelog_end
2020-11-13 15:59:34 +01:00
Andreas Herrmann
1964920ae1
Trigger auth failure (#7952)
* Factor out oauth2 test server body into a class

changelog_begin
changelog_end

* move start into class

* mutable authorized parties

* add function sfor party authorization

* manage party authorization in trigger fixture

* enable deleting cookies

* Add access denied test cases

* Track readAs and actAs claims

* Enable ignored auth tests

* fmt

* add method comments

* Update triggers/service/auth/src/main/scala/com/daml/oauth/server/Server.scala

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update triggers/service/auth/src/main/scala/com/daml/oauth/server/Server.scala

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Explain party authorization modes

* inline expected status codes

* fix closing brace

* use shouldBe instead of should equal

* Use shouldBe instead of should equal

* Explain revoking access and deleting cookies

* foreach requires a total function

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-11-12 23:09:30 +00:00
Moritz Kiefer
07d5a19d50
Cleanup trigger service endpoints (#7950)
* Cleanup trigger service endpoints

fixes #6333

changelog_begin

- [Trigger Service]

  Endpoints have been rearranged to be more consistent:

  | New endpoint              | Old endpoint       | Functionality                |
  |---------------------------|--------------------|------------------------------|
  | GET `/v1/triggers`        | `/v1/list`         | List triggers                |
  | POST `/v1/triggers`       | `/v1/start`        | Start trigger                |
  | GET `/v1/triggers/:id`    | `/v1/status/:id`   | Trigger status               |
  | DELETE `/v1/triggers/:id` | `/v1/triggers/:id` | Stop/delete trigger          |
  | POST `/v1/packages`       | `/v1/upload_dar`   | Upload DAR                   |
  | GET `/livez`              | `/v1/health`       | liveness check               |

changelog_end

* Fix integration tests

changelog_begin
changelog_end
2020-11-12 11:23:28 +01:00
Stephen Compall
b35c9fcb3d
slow down trigger on RESOURCE_EXHAUSTED failures (#7820)
* restate the submit stage as a Flow and derived Sink

* take submit out of the trigger-to-submit flow

* type for the failures produced directly by command submission

* directly connect the msgSource failure queue to the submitter output

* parens

* slow down submission as we exceed max parallel submissions

* restricting alterF so it will be usable with ConcurrentMap

* disable buffer for the delay

* split out the delay function

* drafting a retry loop

* degenerate test for retry loop, factoring the forAllFuture utility

* map input to retrying properly

* make retrying accessible to tests

* test happy path and fix off-by-one

* further tests for retrying

* reveal that elements can get lost

* more determinism in test

* let failures block further elements from being attempted

- Previously failures would go into a separate queue, where they awaited expiry
  of their delay and further initial upstream elements were given their first
  tries.  However, closing the upstream could mean that queue was dropped, and
  detecting that situation is not trivial.  So, instead, we don't use a separate
  queue.

* plug retrying into the trigger submission flow

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* remove throttle; pendingCommandIds may leak

* report random parameter on failure

* revert comment about throttling

* explanation for fail in the error queue

- suggested by @cocreature; thanks
2020-11-10 21:36:48 +00:00
Andreas Herrmann
70d7c97e7f
Handle authorization failure in the auth middleware and trigger service (#7920)
* oauth error response

* oauth server parties parameter

* implement request denial in oauth test server

* Test oauth test server for access denied

* test unauthorized party on auth middleware

* Handle OAuth2 login flow error response in auth middleware

* Forward login error to auth middleware client

* fmt

* fix server test

* fmt

* Handle login failure in trigger service

* Test unauthorized trigger start

* Cleanup authCallback

* Update authentication specification

- The auth middleware accepts an arbitrary callback URI on /login
- The auth middleware will forward OAuth2 authorization failures to the
  client service.

changelog_begin
changelog_end

* Update triggers/service/authentication.md

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-11-10 09:27:21 +00:00
Andreas Herrmann
9758b2f85b
Check trigger dao migrations digests (#7908)
* Check the trigger dao migrations digest

Following the example of the corresponding ledger on SQL tests.

The digests had to be updated as both of them had gone out of sync.
The init digest presumably due to the change in #7226 and the one for
adding the access token during review of #7890.

changelog_begin
changelog_end

* define abstract migrations test

* Use abstract migrations test in trigger service tests

* use abstract migrations test in ledger on SQL

* Retain check for number of .sql resources

* Factor out the hash-migrations script

* Consistent shell settings

Addressing review comment

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-06 09:20:32 +00:00
Andreas Herrmann
c61abf0ab0
newtype wrapper for access token in trigger service (#7906)
changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-11-05 14:07:04 +00:00
Moritz Kiefer
31a3f25800
Add a --port-file option to the trigger service (#7907)
fixes #7097

changelog_begin

- [Triggers] The trigger service now has a `--port-file` option
  matching the corresponding option in the JSON API.

changelog_end
2020-11-05 15:03:34 +01:00
Moritz Kiefer
edc243963c
Expose Event from the high-level trigger API (#7899)
This allows you to write somewhat useful update functions since you
can inspect the transactions. Created and Archived are kept abstract
but we expose fromCreated/fromArchived to interact with them.

fixes #6968

changelog_begin

- [DAML Triggers] The `Daml.Trigger` module now reexports `Event`
  which avoids having to import `Daml.Trigger.LowLevel` for implementing
  a non-trivial `updateState` function.

changelog_end
2020-11-05 15:02:17 +01:00
Moritz Kiefer
2a3e8095b8
Support multiple --dar options in the trigger service (#7903)
Limiting this to a single one makes little sense and while you can
work around it by uploading more packages, that can be annoying during
development.

fixes #6332

changelog_begin

- [Triggers] The trigger service now accepts multiple `--dar`` options.

changelog_end
2020-11-05 14:01:06 +01:00
Andreas Herrmann
4f92ed13ed
Store access token in trigger dao (#7890)
* Test all four trigger service configurations

- non-authenticated with in-memory backend
- non-authenticated with database backend
- authenticated with in-memory backend
- authenticated with database backend

changelog_begin
changelog_end

* add access token to trigger dao

* Simplify readRunningTriggers

* fmt

* Rename V2__Add_access_token.sha25 to V2__Add_access_token.sha256

* Apply suggestions from code review

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-11-05 09:45:40 +00:00
Moritz Kiefer
4fe9196ceb
Improve error on query failures in toACS (#7898)
Getting a pattern match failure here if you accidentally pass in an
archived contract is a bit cruel and now that this is DAML Script
where we can handle the failure, we can do better.

changelog_begin
changelog_end
2020-11-05 10:38:54 +01:00