Commit Graph

255 Commits

Author SHA1 Message Date
Stephen Compall
664a0c0076
add Action to high-level trigger updateState (#7621)
* add ActionState to the standard library

* use 1 ActionState, 1 get, 1 put in low-level trigger library

* introduce TriggerStateA for updateState

* fix tests and examples for new updateState signature

CHANGELOG_BEGIN
- [Triggers] The ``updateState`` function now returns a ``TriggerStateA``.  This
  is an action like ``TriggerA``, but doesn't permit emitting commands.  Instead
  of taking the state as an argument and returning a new state, you can
  manipulate the state with ``get``, ``put``, and ``modify``.  Any existing
  ``updateState`` can be ported by replacing ``s -> expr`` in the lambda
  expression with ``-> modify $ \s ->``, and then made to look nicer from there
  as desired.
  See `issue #7621 <https://github.com/digital-asset/daml/pull/7621>`__.
CHANGELOG_END

* some DAML docs for updateState and TriggerStateA
2020-10-09 13:56:24 -04:00
Andreas Herrmann
8b9c237031
map ledger-api claims to scopes for auth0 (#7629)
* Define mapping from claims to scope

changelog_begin
changelog_end

* Update auth0 instructions

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-10-09 18:43:56 +02:00
Stephen Compall
c296e22a80
remove submitCommands, ActionTrigger from high-level Trigger API (#7619)
- for parity with what was allowed pre-#7456

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-09 07:34:33 +00:00
Stephen Compall
a1d8e8dc33
replace commands-in-flight argument to trigger rule with getCommandsInFlight action (#7600)
* add getCommandsInFlight as a TriggerA action

* immediately update commandsInFlight on emitCommands

- delay until the rule had returned made sense before submitCommands was launched
  immediately; this also makes `getCommandsInFlight` more sensible

* remove commands-in-flight argument from high-level Trigger rule

CHANGELOG_BEGIN
- [Triggers] The "commands in flight" or ``Map CommandId [Command]`` argument has been
  removed from high-level trigger ``rule`` functions; instead, the current
  commands-in-flight can be retrieved with the new ``getCommandsInFlight`` function, which
  can be done immediately at the beginning of the rule's ``do`` block to preserve exact
  existing trigger behavior.
  See `issue #7600 <https://github.com/digital-asset/daml/pull/7600>`__.
CHANGELOG_END

* remove commands-in-flight argument from trigger tests

* update doc examples and copy in compatibility

- compatibility change will most likely entail another flag day in compatibility tests,
  replacing last_pre_7456_trigger_version

* update doc text for getCommandsInFlight

* test that getCommandsInFlight gets updated during the rule

* flag day for trigger compatibility tests

sdk-version: 1.7.0-snapshot.20201006.5358.0.0c1cadcf
File:     src/CopyTrigger.daml
Hidden:   no
Range:    55:11-55:19
Source:   typecheck
Severity: DsError
Message:
  src/CopyTrigger.daml:55:12: error:
  • Couldn't match type ‘TriggerA ()’ with ‘() -> TriggerA ()’
  Expected type: Party
-> DA.Next.Map.Map CommandId [Command] -> () -> TriggerA ()
  Actual type: Party -> ACS -> () -> TriggerA ()
  • In the ‘rule’ field of a record
  In the expression:
  Trigger
  {initialize = \ _acs -> (), updateState = \ _acs _message () -> (),
  rule = copyRule, registeredTemplates = AllInDar, heartbeat = None}
  In an equation for ‘copyTrigger’:
  copyTrigger
  = Trigger
  {initialize = \ _acs -> (), updateState = \ _acs _message () -> (),
  rule = copyRule, registeredTemplates = AllInDar, heartbeat = None}

* match docs on TriggerAState to current usage of these fields

* remove emittedCommands, as commandsInFlight is now kept up-to-date

* zoomed from where?

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-10-08 12:03:56 -04:00
Stephen Compall
dc204a37a7
remove known command ID from trigger state once Transaction and Completion message are seen (#7598)
- no assumption about order

- multiple transactions/completions are treated as one

CHANGELOG_BEGIN
CHANGELOG_END
2020-10-07 18:32:49 +00:00
Stephen Compall
ff706d5762
Trigger monad's Submit emits a command ID instead of accepting one (#7587)
* make TriggerF's Submit emit a command ID instead of accepting one

- LowLevel updated

- interpreter in Scala updated, preserving Message semantics as closely as possible

* remove nextCommandId and all management from high-level Trigger API implementation

* a note about TriggerF and simulateRule

* port submitCommands calls from tests

CHANGELOG_BEGIN
- [Triggers] The CommandIds as accessed from trigger DAML code are now exactly the command
  IDs used in command submission to the ledger; as such, they will vary randomly from run
  to run of the trigger rule.  To enable this, the low-level ``submitCommands`` function
  no longer accepts a command ID, instead returning one; there is no change to the
  corresponding high-level ``emitCommands`` function, so high-level triggers should only
  see improved and easier-to-understand logging.
  See `issue #7587 <https://github.com/digital-asset/daml/pull/7587>`__.
CHANGELOG_END

* port Scala-side assertion of trigger test

* rename commandIdMap to commandIdsUsed

* test that command IDs are like the ones we expect from the runner
2020-10-07 08:34:41 -04:00
Andreas Herrmann
8dee7ac9fc
Make trigger service auth middleware /login endpoint compatible with Auth0 (#7565)
* Make /login endpoint compatible to auth0

- Make the authorization and token endpoints configurable
- Use `application/x-www-form-urlencoded` as specified in
  https://tools.ietf.org/html/rfc6749#section-4.1.3
- Check the status code of the token endpoint response
- Fix the type of the token's `expires_in` field
  (`Int` instead of `String)

changelog_begin
changelog_end

* Add Auth0 testing instructions

* Use native application type on Auth0

* scope to claims mapping todo note

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-10-05 13:44:46 +00:00
Stephen Compall
0e71a2d3d2
interpret allowed updates and other actions in a free Trigger monad, like Script (#7456)
* conservatively move daml-script, trigger SValue interpreters to common library

* introduce expect and JavaList pattern for converters

* clean up trigger Converter Command interpretation

* add Church Free monad

* add an action language for trigger updates

* add expectE to remove some of the joins

* convert more of the converters to expect

* tool for unrolling Free/Roll

* split handleStepResult up and clean up its pattern

* handleStepFreeResult to interpret TriggerF

* replace Free Church with Pure/Roll free from Script

* newtype for ActionTrigger

* replace update in low-level Trigger with Free TriggerF

* submit one Commands at a time

* boolean blindness strikes again

* log missed TriggerF steps

* comment actual Submit contents

* match #7501 fromPureSExpr sig change in 00b80b8ea3

* avoid using forwardPort in runTrigger

* push State back into DAML, so it can be excluded from the action list

* push Message back into DAML, unifying the action language for initialState and update

* bringing TriggerF into initial state

* really add TriggerF into initial state, with all ports, tested

* add ActionTrigger class, express initialState in its terms

* add all TriggerF actions to existing TriggerA

* Trigger.rule will no longer have Time argument

* rename getS, setS to get, put, matching C.M.T.State from transformers

* make high-level Rule evaluate to the underlying TriggerF sequence

* Assert's testRule doesn't have a transform yet

* move DamlTuple2 to common converter library

- suggested by @cocreature; thanks

* combine the two Frees, provide from Script

* remove time argument from integration tests

CHANGELOG_BEGIN
- [Triggers] The ``Time`` argument was removed from the trigger rule function; instead, it
  can be fetched within the ``TriggerA`` ``do`` block by ``getTime``, as with ``Update``
  and ``Scenario``.  The ``LowLevel`` trigger interface has been redesigned; such triggers
  need to be rewritten or ported to high-level triggers.
  See `issue #7456 <https://github.com/digital-asset/daml/pull/7456>`_.
CHANGELOG_END

* add trigger rule simulator to support Assert module

* missed new Free module

- left in script per @cocreature

* remove retract as we ended up using foldFree for that purpose instead

- suggested by @cocreature; thanks

* throw ConverterException instead of RuntimeException

- suggested by @cocreature; thanks

* remove Time argument from coin-upgrade-trigger

* port trigger service tests

* port trigger scenario test

* put TriggerSetup and TriggerRule into LowLevel.Trigger instead of unboxed Free

- suggested by @cocreature; thanks

* remove Time argument from trigger compatibility test

* submit commands as soon as each `emitCommands` is sequenced

- we still collect a list, but only for tracking commandsInFlight

* filter out compatibility tests for triggers before now

* remove commented imports, libraries from new shared converter

* make the TriggerF interpreter tail-recursive

* remove unused compatibility trait

* add back new state logging

* remove refactoring comment

* rewrite some LowLevel initialStates in do

* hide Daml.Script.Free from docs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* remove forwardPortInitialState

- suggested by @cocreature; thanks

* manually port low-level updates

- suggested by @cocreature; thanks

* remove forwardPort

- suggested by @cocreature; thanks

* fail faster on unrecognized TriggerF

- suggested by @cocreature; thanks

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-10-02 14:18:13 -04:00
Moritz Kiefer
e6c43c798f
Document cookie passing in trigger service auth (#7554)
This has come up 2 times now so seems sensible to point this out
explicitly.

changelog_begin
changelog_end
2020-10-02 14:46:53 +02:00
Andreas Herrmann
c5abcece56
trigger service auth /auth endpoint (#7545)
* Factor out request bodies

Addressing review comment
https://github.com/digital-asset/daml/pull/7519#discussion_r497321689

* Implement /auth endpoint

changelog_begin
changelog_end

* /auth check the required claims

* Factor out middlewareUri

* fmt

* less implicit variables

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-10-01 15:10:38 +00:00
Andreas Herrmann
5a4f1d5d18
trigger service auth /login endpoint (#7519)
* trigger service auth /login endpoint

changelog_begin
changelog_end

* Configure client id and secret through environment

Addressing review comment
https://github.com/digital-asset/daml/pull/7519#discussion_r497303019

* Single flag for OAuth2 server URI

Addressing review comment
https://github.com/digital-asset/daml/pull/7519#discussion_r497301985

* Explain redirectUri in comments

Addressing review comment
https://github.com/digital-asset/daml/pull/7519#discussion_r497305958

* TODO configurable redirect URI

Addressing review comment
https://github.com/digital-asset/daml/pull/7519#discussion_r497318955

* Verify that cookie holds a token

Addressing review comment https://github.com/digital-asset/daml/pull/7519#discussion_r497328320

* fmt

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-09-30 11:38:54 +00:00
Moritz Kiefer
00b80b8ea3
Separate off-ledger ond on-ledger speedy (#7501)
* Separate off-ledger ond on-ledger speedy

changelog_begin
changelog_end

* Add OnLedgerBuiltin abstract class

changelog_begin
changelog_end

* fix typo

changelog_begin
changelog_end

* Fix borked rebase

changelog_begin
changelog_end

* Remove unscoped withOnLedger

changelog_begin
changelog_end
2020-09-29 10:05:45 +00:00
Remy
a899e5e4e6
Speedy: Check language version as part of Speedy compilation. (#7440)
Additionnally
+ move allowedLanguageVersion inside compiler config
+ add missing catch error insde
  ConcurrentCompiledPackages#addPackageInternal
+ implement EngineConfig method to easily produce a Compiler.Config

CHANGELOG_BEGIN
CHANGELOG_END
2020-09-22 12:43:48 +02:00
Stephen Compall
a670e68c72
add 'alter' function to DA.Map (#7427)
* add 'alter' function to DA.Map

CHANGELOG_BEGIN
CHANGELOG_END

* swap arguments for alter

- suggested by @cocreature; thanks

* alter moved to stdlib, remove from trigger lib

* use markdown formatting for alter doc

- suggested by @sofiafaro-da; thanks

* fix code block formatting

- suggested by @sofiafaro-da; thanks
2020-09-17 16:46:37 +00:00
Stephen Compall
1926ddbe9d
avoid ACS linear contract ID lookups in Trigger library (#7425)
* add Ord (ContractId a) instance

* change inner list in activeContracts to map

* adapt Internal module to new map

* support rest of Trigger daml lib

* rename adjust to alter, its Data.Map name

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* distribute type arguments for ACS activeContracts

- suggested by @cocreature; thanks

* swap arguments for alter

- suggested by @cocreature; thanks

* reformat groupActiveContracts signature

- suggested by @cocreature; thanks
2020-09-17 15:31:03 +00:00
Moritz Kiefer
b6673b6b4d
Add design doc for authentication in the trigger service (#7394)
* Add design doc for authentication in the trigger service

This is a draft of how I currently imagine authentication in the
trigger service to work. Since the authentication middleware has to be
pluggable in the end anyway we need public documentation in the end
anyway and I find this much easier to manage than a google doc outside
of the repo which just never sees updates.

changelog_begin
changelog_end

* Update triggers/service/authentication.md

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-09-16 15:57:26 +02:00
Stephen Compall
09014dc637
use GenMaps for trigger ACS (#7365)
* use GenMaps for trigger ACS

* Next removed for 1.dev

* temp port trigger test code to 1.dev only

* run trigger tests on 1.dev only

* move pending back to TextMap

* include trigger service in the 1.dev test lineup

- it takes >2min, so shouldn't be permanent

* add Ord TypeRep and Ord TemplateTypeRep when possible

* swap names in Internal to reduce the diff

* try to enable cpp for triggers compilation

$ bazel build //triggers/daml:daml-trigger-1.dev
<snip>
File:     daml/Daml/Trigger/Internal.daml
Hidden:   no
Range:    103:-1-103:-1
Source:   CPP
Severity: DsError
Message:  22 in hpp-0.6.1:Hpp.CmdLine
File:     daml/Daml/Trigger.daml
Hidden:   no
Range:    103:-1-103:-1
Source:   CPP
Severity: DsError
Message:  22 in hpp-0.6.1:Hpp.CmdLine
ERROR: Creation of DAR file failed.
<snip>

* remove problematic options for invoking cpp

hpp: Couldn't open input file: -Werror
CallStack (from HasCallStack):
  error, called at src/Hpp/CmdLine.hs:103:22 in hpp-0.6.1:Hpp.CmdLine

* enough cpp so default and 1.dev triggers compile

* cpp needed for docs as well

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* return trigger service to testing against sdk default lf version

* run trigger integration test against sdk default and LF 1.dev

* return trigger scenario test to SDK default LF version

* avoid import warnings in trigger lib

* Windows manifests a missing file differently. Hilarious
2020-09-15 12:30:23 -04:00
Moritz Kiefer
c0b6851abe
Generate hoogle docs for daml script/triggers (#7371)
* Generate hoogle docs for daml script/triggers

This PR switches over the documentation generation for daml script and
daml triggers to the multi-page format we already use for the standard
library and extends it to also generate hoogle documentation.

All 3 hoogle files are combined in a single hoogle_db.tar.gz archive.

Since the location in the multi-page format is different, I’ve added
redirects.

I verified locally, that I can generate the hoogle database and that
the links point to the right places.

changelog_begin
changelog_end

* Fix baseurl for daml-stdlib

changelog_begin
changelog_end
2020-09-10 15:58:15 +02:00
Moritz Kiefer
3171bb588a
Migrate trigger assertion library to DAML Script (#7318)
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).

changelog_begin

- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.

changelog_end
2020-09-04 13:48:50 +02:00
Stephen Compall
0b72b6f716
more trigger service logs; tagged LoggingContext (#7205)
* get a LoggingContext into the TriggerRunnerImpl

* make some implicits more implicitly scoped and explicitly ascribed

* make some private/final markings

* most of JsonFormat[Identifier] is in companion

* experimental LoggingContext with phantom type

* ActorContext#log isn't really doing that much

* more details of LoggingContextOf

* make LoggingContextOf compile

* add trigger message logging, yet without context

* fix parent compile errors

* use Config as the phantom for its own logging extensions

* LocalDateTimeFormat cleanup

* switch TriggerRunner to contextual logging

* add trigger definition ID to logs

* log trigger-submitted commands, fix trigger test compile

* log trigger stopping and DAR uploads

* add context to PostStop/PreRestart logs

* add changelog

CHANGELOG_BEGIN
- [Triggers] More detailed logging of trigger actions and trigger service actions.
  See `issue #7205 <https://github.com/digital-asset/daml/pull/7205>`_.
CHANGELOG_END

* missed copyright header

* switch to Unit, scala/bug#9240 fixed
2020-09-03 17:47:09 +00:00
Samir Talwar
2b3cf1b3c0
ledger-api-client: Rename maxInboundMessageSize to maxInboundMetadataSize. (#7290)
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.

CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
  to ``maxInboundMetadataSize``, to match the builder Netty channel
  builder. It was incorrectly named ``maxInboundMessageSize``, which is
  a different channel property that configures the maximum message size,
  not the header size.
CHANGELOG_END

* ledger-api-client: Introduce a `maxInboundMessageSize` config property.

We use this a lot; easier if it's in the configuration.

CHANGELOG_BEGIN
- [Scala Bindings] Replace the
  ``LedgerClientConfiguration.maxInboundMessageSize`` property with a
  new one that represents the maximum size of the response body.
CHANGELOG_END
2020-09-02 08:41:34 +00:00
Moritz Kiefer
1df6ff49e3
Add a dummy OAuth2 authorization server (#7250)
* Add a dummy OAuth2 authorization server

This is intended for use in the trigger service integration tests not
as an artifact that we ship to users.

changelog_begin
changelog_end

* Address review feedback

changelog_begin
changelog_end
2020-08-28 17:28:06 +02:00
Moritz Kiefer
6e4f101aad
Remove ref-ledger-authenticator from trigger service (#7226)
* Remove authentication service from trigger service

This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).

While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.

changelog_begin

- [Trigger Service] Parties are now specified in request bodies as
  opposed to via HTTP Basic auth. This is done in preparation for
  running the trigger service against authenticated ledgers.

changelog_end

* Fix integration tests

changelog_begin
changelog_end
2020-08-27 15:36:59 +02:00
Remy
89770cf1c1
Speedy: make the field names from SRecord immutable. (#7225)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-25 17:02:27 +02:00
Stephen Compall
2369a3bbae
use sandbox-style flyway migrations for trigger service (#7092)
* depend on flyway in trigger service; duplicate FlywayMigrations from sandbox

* bring in postgres/h2 detection, other data-source elements needed; use scalalogging

* use DataSource instead of DriverManager for DbTriggerDao, pass to flyway too

- initialization order isn't quite right yet

* hardcode postgres in triggers flyway path

* found an init order that doesn't fail in tests

* better pools for production, and don't forget to close them

* move most init SQL to first migration file

* make logHandler implicit again

- seems to have been an accidental implication of #6334

* move remaining create to V1__Init.sql

* use Prod poolSize

* prove that PostStop signals are never seen (in tests)

* move all the shutdown to Stop message

* remove dead code in DbFlywayMigrations

* always "initialize" database

CHANGELOG_BEGIN
- [Trigger Service] The database format has changed to allow migrations in
  future releases.  Databases are always initialized or migrated to the current
  version on start, so use of ``--init-db`` is no longer required.
  See `issue #7092 <https://github.com/digital-asset/daml/pull/7092>`__.
CHANGELOG_END

* it should always be obvious when there is no escape

* remove logs from experiments

* remove receiveSignal PostStop

* remove unused imports
2020-08-20 18:52:37 +00:00
Samir Talwar
30564a7c90
ledger-api-client: Add test cases for LedgerClient. (#7195)
* ledger-api-client: Add integration tests for the simple stuff.

* sandbox-common: Make `SandboxFixtureWithAuth` a mixin.

This makes it useful with `SandboxNextFixture` as well as
`SandboxFixture`.

Also, add types to non-private fields and methods, and make more fields
protected rather than public.

* ledger-api-client: Add tests to make sure the token is passed through.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox-common: Tokens are for auth, not auth.
2020-08-20 15:00:32 +00:00
Remy
e54c182032
Engine: move check of valide packages inside MutableCompiledPackages (#7186)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-19 19:37:53 +02:00
Moritz Kiefer
d4d0419c64
Use TO_TEXT_CONTRACT_ID in Show instance of ContractId (#7153)
fixes #7114

This PR changes the Show instance of ContractId and flips the switch
on triggers and DAML Script to run in off-ledger mode.

It also adds a test that for DAML Script we actually get back the
correct contract id.

There is a bit of a design decision here in how we want to print
contract ids, so let me list the options I considered. $cid will stand
for the actual cid and all options are wrapped in markdown inline
code.

1. `"$cid"`. Indistinguishable from string. Suggests that there might
be an IsString instance for ContractId.
2. `<$cid>`. Matches the dummy `<contract-id>` but it’s not a dummy so
I don’t think matching that is benefitial.
3. `$cid`. Easy to spot (contract ids start with # and have no
spaces), clearly not a string but might look slightly weird.

changelog_begin

- [DAML Script/DAML Triggers] When using DAML-LF 1.dev, the `Show` instance of `ContractId` will now display the actual contract id instead of a dummy `<contract-id>` value. Note that this only applies to DAML Script and DAML Triggers not to ledger code.

changelog_end
2020-08-17 17:06:24 +02:00
Stephen Compall
1737907415
add --address option to trigger service (#7090)
* factor --address, --http-port, --port-file options from http-json to cli-opts

- enabling reuse in trigger service

* use cli-opts for address and http-port options in Trigger service

* mark ServiceConfig and some defaults private

* use --address option to set up server

* document Setter

* test --address option is parsed

* missing (c) headers

* add changelog

CHANGELOG_BEGIN
- [Trigger Service] Accepts a new ``--address`` option to listen for HTTP connections on
  interfaces other than localhost, such as ``0.0.0.0`` for all addresses.
  See `issue #7090 <https://github.com/digital-asset/daml/pull/7090>`__.
CHANGELOG_END
2020-08-12 13:50:00 -04:00
Stephen Compall
7d5d6e125f
ensure /v1/status returns all results for restarted triggers (#7052)
* test only 'restart triggers with update errors' in CI, 100 times

CHANGELOG_BEGIN
CHANGELOG_END

* build fewer things and try 1000 runs

* test restart after shutdown, 200 times

* distrust the log

* report which withTriggerService is being used to provide log dividers

* log when TriggerStarting, TriggerStarted received in 'starting' state

- In mainline, we only add logs when we add triggers in the 'running'
  stat in Server.apply.  Let's see if we aren't sending these messages
  to the `Server.start` actor when it is still in the 'starting' loop

* log same thing in starting and running states

* revert build experiment and extra logs

- keep the fixture location log, as it's a very useful divider when
  parsing test results
2020-08-11 14:55:44 +00:00
Stephen Compall
0857c28874
turn on NonUnitStatements in trigger runner, service main code (#7079)
* turn on NonUnitStatements in trigger runner, service main code

* add some friendly discards

* add a not-so-friendly discard

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* one more discard to clarify an Await.ready's type

* explain unbind issue
2020-08-11 14:30:11 +00:00
Moritz Kiefer
39085493c8
Make ApplicationId in DAML Triggers configurable (#7075)
fixes #7030

This deliberately ignores the trigger service. The main reason for
setting the ApplicationId at the moment is authentication and this is
still very WIP in the trigger service, so I don’t think it makes sense
to add this in some form to the API until that has settled.

changelog_begin

- [DAML Trigger] You can now configure the application id via
  `--application-id` in `daml trigger`. This is primarily useful if
  you run with authentication.

changelog_end
2020-08-10 18:15:16 +02:00
Stephen Compall
d2ee532949
remove more unused definitions, params, args from triggers Scala code (#7032)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-05 19:06:22 +00:00
Rohan Jacob-Rao
765cd181b5
Trigger service test: Increase timeout for trigger to start running after service shutdown (#6986)
changelog_begin
changelog_end
2020-08-04 18:11:06 +00:00
Stephen Compall
c3e79878ff
remove unused definitions, params, args from ledger API Scala code (#6985)
* remove unused definitions, params, args from ledger API Scala code

CHANGELOG_BEGIN
- [Ledger API] withTimeProvider removed from CommandClient; this method
  has done nothing since the new ledger time model was introduced in
  1.0.0.  See `issue #6985 <https://github.com/digital-asset/daml/pull/6985>`__.
CHANGELOG_END

* percolate withTimeProvider and label removal elsewhere
2020-08-04 18:02:19 +00:00
Stephen Compall
4ba9ae4e2b
remove unused definitions, params, args from triggers Scala code (#6983)
CHANGELOG_BEGIN
CHANGELOG_END
2020-08-04 16:50:32 +00:00
Stephen Compall
3196ef7575
fix flaky 'restart triggers after shutdown' test (#6926)
* reenable 'restart triggers after shutdown'

CHANGELOG_BEGIN
CHANGELOG_END

* wait for everything to shut down before completing a withTriggerService fixture

- similar to a change to HttpServiceFixture.withHttpService in #4593,
  but without the suppression of shutdown errors

* label the WithDb tests

* in CI, test only 'recover packages after shutdown', 50 times

* experiment: Process#destroy appears to be async

* is it in the in-between period?

* partial -> total

* replace some booleans with assertions for better error reporting

* make triggerLog concurrent

* close channel and file in other error cases for port locking

- suggested by @leo-da; thanks

* use port locking instead of port 0 for trigger service fixtures

* destroy one service at a time

* missed continuation in build script

* use assertion language for "restart triggers with update errors"

* Revert "is it in the in-between period?"

This reverts commit 211ebfe9d2.

* use better assertion language for "restart triggers with update errors"

* restore full CI build
2020-08-04 08:15:08 -04:00
Moritz Kiefer
5ceeb476c8
Remove duplicated tls config parsing from DAML Script/Triggers (#6947)
This was not only unnecessarily duplicated, it also had a bug where
`--crt` behaved like `--pem` instead of setting the cert chain.

I didn’t add new tests since it seems like the wrong place to test
config parsing of a library. We do have tests for TLS in general for
both DAML Script and DAML Triggers.

changelog_begin
changelog_end
2020-07-31 18:25:18 +02:00
Rohan Jacob-Rao
e6cf6d0d51
Trigger service tests: sequence toxiproxy setup before starting sandbox (#6939)
This should avoid potential contention for ports.

changelog_begin
changelog_end
2020-07-30 15:32:46 -04:00
Rohan Jacob-Rao
91b174da91
Trigger service tests: Remove auth service process in trigger service fixture (#6928)
This code has a few problems: the auth service test mode is not enabled, the
admin ledger should be separate from main ledger if auth is enabled, and the
retry strategy not executed.

The auth service is not used in these tests yet and we have an example
of how to do it correctly in the auth service fixture (and auth service
client tests). We can try to reuse that fixture instead or copy the
code over when necessary.

changelog_begin
changelog_end
2020-07-29 23:29:04 +00:00
Rohan Jacob-Rao
4c76cf98ee
Trigger service tests: Make sure toxiproxy server is running before connecting the client (#6927)
* Trigger service tests: Make sure toxiproxy server is running before connecting client

changelog_begin
changelog_end

* Undo infuriatingly wrong camel case
2020-07-29 23:00:43 +00:00
Samir Talwar
99463577af
triggers + ports: Lock free ports before binding. (#6911)
* triggers: Use `FreePort.find()`.

* ports: Move `LockedFreePort` from postgresql-testing for reuse.

* triggers: Use `LockedFreePort` to avoid race conditions.

* ports + triggers: Move common port testing into the ports library.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-29 14:29:55 +00:00
Samir Talwar
98de16606d
daml-on-sql: An elegant Main class, for a more civilized age. (#6829)
* daml-on-sql: Pull out a new `Main` object that wraps sandbox-classic.

CHANGELOG_BEGIN
CHANGELOG_END

* daml-on-sql: Fail if a JDBC URL is not provided or not for PostgreSQL.

* sandbox-classic: Rename the conformance test H2 database.

* daml-on-sql + sandbox-classic: Report configuration errors cleanly.

This means letting `ProgramResource` catch the errors, log, and exit.

* daml-on-sql: Change the name logged on startup.

* daml-on-sql: Change the default participant ID.

* sandbox-common: Give the ledger name its own tagged string type.

* sandbox-classic: Generate random ledger IDs using the ledger name.

* daml-on-sql: Remove the banner, replacing it with a blank line.

* daml-on-sql: Enable strong seeding by default.

And weak seeding in the conformance tests.

* sandbox-classic: Move the ledger name to a separate parameter.

It's not really configurable.

* sandbox-classic: Move LedgerName from sandbox-common.

* daml-on-sql: Remove "-participant" from the participant ID.

* daml-on-sql: Use `Name` where possible.

* daml-on-sql: Make the ledger ID mandatory.

* Revert "sandbox-classic: Move LedgerName from sandbox-common."

This reverts commit 0dad1584a7.

* daml-on-sql: Print "DAML-on-SQL" in the CLI help, not "Sandbox".

* daml-on-sql + sandbox + sandbox-classic: Split out custom CLI parsing. (#6846)

* participant-state: Simplify naming the seeding modes.
2020-07-24 18:54:19 +00:00
Rohan Jacob-Rao
4973bd3573
Trigger service: rest of auth service client (#6804)
* Method for service account request

* Extend domain and factor out sa/secure

* Methods to list service accounts and poll for service account

changelog_begin
changelog_end

* Run auth client tests 20 times

* Revert "Run auth client tests 20 times"

This reverts commit 5264ce6f30.

* Assert creds list for new service account is empty

* Request credential method

* Method to get a new credential id

* Method for fetching credential

* Login method (for ledger access token)

* Make authorize style consistent with rest

* Remove redundant test (subset of second test)

* Run auth client tests 20 times

* Fail future for http error responses and refine retry strategy

* Scalafmt

* Revert "Run auth client tests 20 times"

This reverts commit 2568dc6b47.

* Test getting service account before request
2020-07-21 21:26:49 +00:00
Samir Talwar
adcbfb45a1
sandbox-classic: Move the configuration from sandbox-common. (#6810)
* sandbox-common: Move the ledgerIdMode up in the configuration.

* sandbox-classic: Move the configuration from sandbox-common.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix miscellaneous warnings caught by IntelliJ IDEA.
2020-07-21 14:43:53 +00:00
Stephen Compall
4355406259
add more scalac 2.12 warnings (#6798)
* set many extra scalac -Xlint options for all Scala projects

CHANGELOG_BEGIN
CHANGELOG_END

* move NoCopy to its own file

package.scala:18: warning: it is not recommended to define classes/objects inside of package objects.
If possible, define trait NoCopy in package data instead.
  trait NoCopy {
        ^

* move more traits, classes, and objects to proper packages

- note that `package` is itself a scoping construct, so if your reason
  is the apparent aesthetic of placing a bunch of things in one `package
  object`, that is easily remedied by deleting the `object` keyword

* fix some type-parameter-shadow warnings

- I'm generally in favor of sensible name-shadowing, following the
  "deliberately hide variables that should not be accessed here" school
  of thought.  But I think type name shadowing isn't quite as valuable
  and more likely to confuse than general variable shadowing, so have
  experimentally linted it out.

  Example warning:

EventsTableFlatEventsRangeQueries.scala:11: warning: type parameter
 Offset defined in trait EventsTableFlatEventsRangeQueries shadows class
 Offset defined in package v1. You may want to rename your type
 parameter, or possibly remove it.
private[events] sealed trait EventsTableFlatEventsRangeQueries[Offset] {
                                                               ^

* fix more package-object-classes warnings

* fix an inaccessible warning

ContractsService.scala:197: warning: method searchDb in class ContractsService references private class ContractsFetch.
Classes which cannot access ContractsFetch may be unable to override searchDb.
  def searchDb(dao: dbbackend.ContractDao, fetch: ContractsFetch)(
      ^

* enable -Xlint:infer-any

- continuing the saga of #6116, #6132

* enable -explaintypes for more detailed type errors

* missed header for NoCopy; probably should have left it in the package file

* misspelling in comment

* revert -Xlint:doc-detached

- there are a lot of these fixes, and they are noisy, so shifting to a
  separate PR
- thanks to @leo-da for pointing out
2020-07-21 08:18:01 -04:00
Moritz Kiefer
c524aee995
Bump timeouts in trigger auth service tests (#6797)
tested via --runs_per_test=8 and seems to be sufficient now.

changelog_begin
changelog_end
2020-07-20 17:06:45 +00:00
Rohan Jacob-Rao
9e979ced9a
Trigger service: Introduce auth service client and test fixture (#6768)
* WIP Call authorize before start trigger

* Intro auth service client

* Auth service client test setup

CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt

* Use Spray JSON format and slightly better uri handling

* Get auth client test to work and fiddle with uris

* Remove changes to Server

* Add package for URI

* Format

* Copyrights

* Update sandbox packages

* Use default wallclock time for auth service sandbox
2020-07-20 13:35:14 +00:00
Robert Autenrieth
7ce9748066
Split sandbox code into separate packages (#6695)
* Move public code into daml-integration-api

CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
2020-07-17 17:06:06 +02:00
Rohan Jacob-Rao
619209d0b1
Trigger service: Document steps to replicate auth flow from the command line (#6725)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 15:05:41 -04:00