Commit Graph

212 Commits

Author SHA1 Message Date
Rohan Jacob-Rao
4973bd3573
Trigger service: rest of auth service client (#6804)
* Method for service account request

* Extend domain and factor out sa/secure

* Methods to list service accounts and poll for service account

changelog_begin
changelog_end

* Run auth client tests 20 times

* Revert "Run auth client tests 20 times"

This reverts commit 5264ce6f30.

* Assert creds list for new service account is empty

* Request credential method

* Method to get a new credential id

* Method for fetching credential

* Login method (for ledger access token)

* Make authorize style consistent with rest

* Remove redundant test (subset of second test)

* Run auth client tests 20 times

* Fail future for http error responses and refine retry strategy

* Scalafmt

* Revert "Run auth client tests 20 times"

This reverts commit 2568dc6b47.

* Test getting service account before request
2020-07-21 21:26:49 +00:00
Samir Talwar
adcbfb45a1
sandbox-classic: Move the configuration from sandbox-common. (#6810)
* sandbox-common: Move the ledgerIdMode up in the configuration.

* sandbox-classic: Move the configuration from sandbox-common.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix miscellaneous warnings caught by IntelliJ IDEA.
2020-07-21 14:43:53 +00:00
Stephen Compall
4355406259
add more scalac 2.12 warnings (#6798)
* set many extra scalac -Xlint options for all Scala projects

CHANGELOG_BEGIN
CHANGELOG_END

* move NoCopy to its own file

package.scala:18: warning: it is not recommended to define classes/objects inside of package objects.
If possible, define trait NoCopy in package data instead.
  trait NoCopy {
        ^

* move more traits, classes, and objects to proper packages

- note that `package` is itself a scoping construct, so if your reason
  is the apparent aesthetic of placing a bunch of things in one `package
  object`, that is easily remedied by deleting the `object` keyword

* fix some type-parameter-shadow warnings

- I'm generally in favor of sensible name-shadowing, following the
  "deliberately hide variables that should not be accessed here" school
  of thought.  But I think type name shadowing isn't quite as valuable
  and more likely to confuse than general variable shadowing, so have
  experimentally linted it out.

  Example warning:

EventsTableFlatEventsRangeQueries.scala:11: warning: type parameter
 Offset defined in trait EventsTableFlatEventsRangeQueries shadows class
 Offset defined in package v1. You may want to rename your type
 parameter, or possibly remove it.
private[events] sealed trait EventsTableFlatEventsRangeQueries[Offset] {
                                                               ^

* fix more package-object-classes warnings

* fix an inaccessible warning

ContractsService.scala:197: warning: method searchDb in class ContractsService references private class ContractsFetch.
Classes which cannot access ContractsFetch may be unable to override searchDb.
  def searchDb(dao: dbbackend.ContractDao, fetch: ContractsFetch)(
      ^

* enable -Xlint:infer-any

- continuing the saga of #6116, #6132

* enable -explaintypes for more detailed type errors

* missed header for NoCopy; probably should have left it in the package file

* misspelling in comment

* revert -Xlint:doc-detached

- there are a lot of these fixes, and they are noisy, so shifting to a
  separate PR
- thanks to @leo-da for pointing out
2020-07-21 08:18:01 -04:00
Moritz Kiefer
c524aee995
Bump timeouts in trigger auth service tests (#6797)
tested via --runs_per_test=8 and seems to be sufficient now.

changelog_begin
changelog_end
2020-07-20 17:06:45 +00:00
Rohan Jacob-Rao
9e979ced9a
Trigger service: Introduce auth service client and test fixture (#6768)
* WIP Call authorize before start trigger

* Intro auth service client

* Auth service client test setup

CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt

* Use Spray JSON format and slightly better uri handling

* Get auth client test to work and fiddle with uris

* Remove changes to Server

* Add package for URI

* Format

* Copyrights

* Update sandbox packages

* Use default wallclock time for auth service sandbox
2020-07-20 13:35:14 +00:00
Robert Autenrieth
7ce9748066
Split sandbox code into separate packages (#6695)
* Move public code into daml-integration-api

CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
2020-07-17 17:06:06 +02:00
Rohan Jacob-Rao
619209d0b1
Trigger service: Document steps to replicate auth flow from the command line (#6725)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 15:05:41 -04:00
Gary Verhaegen
8176fb0c8d
fix the the typo (#6723)
```
s/the the /the /
```

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-14 16:55:33 +00:00
Rohan Jacob-Rao
98613d9671
Trigger service: doc explaining planned use of auth service (#6659)
* Add details about validity windows
* Reword description of auth service concepts
* Elaborate on auth flow dealing with token/credential expiry

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-09 14:27:50 -04:00
Rohan Jacob-Rao
30f1bf8d5a
Disable recovery test until I can fix the flakiness (#6654)
CHANGELOG_BEGIN
CHANGELOG_END
2020-07-08 11:17:12 -04:00
Moritz Kiefer
02c59d4f2a
Revert ANF changes and add a testcase for evaluation order (#6645)
* Revert ANF changes and add a testcase for evaluation order

After careful consideration, we decided that the change in evaluation
order that was accidentally introduced by the ANF changes should be
considered a breaking change or arguably even a bug and should not
land in 1.3.0.

Therefore, this PR reverts the following commits:

1. 353d0da6f7
2. a45b51042f
3. 04c7b2af7f
4. a624dd7242
5. b3aab72cee

Other PRs mostly had trivial merge conflicts that I resolved. The two
most interesting ones here are probably

1. https://github.com/digital-asset/daml/pull/6576 which was easy to
   resolve and the change to return SEValue instead of SExpr is still
   nice and useful even if we do not need the guarantees.
2. it https://github.com/digital-asset/daml/pull/6542 which required
   some changes since the constructors changed. If you want to review
   those changes in detail (they are pretty straightforward so not too
   important), it’s probably easiest to check out this PR and run
   ```
   git diff 2cd2a8f2a8
   daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/Compiler.scala
   ```
   to see the diff to the parent commit of the first commit that
   introduced ANF.

changelog_begin
changelog_end
2020-07-08 12:40:12 +02:00
Stephen Compall
4cd419623a
replace traverseU and sequenceU with traverse and sequence (#6594)
* replace traverseU and sequenceU with traverse and sequence

- with -Ypartial-unification on, the extra Unapply typeclass lookup is
  unnecessary

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* limit imports; we only need *> and void
2020-07-07 15:01:58 +00:00
Rohan Jacob-Rao
8261509312
Trigger service: Remove retries when checking trigger ids (#6628)
The running trigger store is now written to synchronously by the server, so there's no need for retry logic.

Also assert equality of trigger ids rather than an arbitrary predicate, as it gives better error messages and we don't use other types of predicates for this function.

CHANGELOG_BEGIN
CHANGELOG_END
2020-07-06 13:48:38 -04:00
Rohan Jacob-Rao
7501805eae
Trigger service: increase timeout for db restart test (#6624)
changelog_begin
changelog_end
2020-07-06 16:29:47 +00:00
Moritz Kiefer
907e4fe678
Fix project versions of DAML Script and DAML Triggers (#6622)
GHC has weird restriction on version numbers which damlc inserits so
we need to use `ghc_version` instead of `sdk_version`. That only makes
a difference for snapshot versions where the `-snapshot.` part is
replaced by `.`.

changelog_begin
changelog_end
2020-07-06 15:07:39 +00:00
nickchapman-da
b3b30ae71d
Speedy issue 6573 (#6576)
* address issue 6573

changelog_begin
changelog_end

* fmt

* push switch from SExpr to SValue to callers of the trigger service
2020-07-02 13:15:00 +01:00
Moritz Kiefer
cf4fcb560b
Fix the package version of daml-trigger and daml-script (#6566)
It makes no sense to keep this at 0.0.1.

changelog_begin

- [DAML Script] The DAML Script library now has the version of the
  corresponding SDK.

- [DAML Trigger] The DAML Trigger library now has the version of the
  corresponding SDK.

changelog_end
2020-07-01 19:06:00 +02:00
Rohan Jacob-Rao
bdaf9aa216
Trigger service: restart triggers in database on server startup (#6556)
* Add optional trigger id to startTrigger

CHANGELOG_BEGIN
CHANGELOG_END

* Read running trigger table in DbTriggerDao

And get rid of traverse type application!
(since we have partial unification on)

* Restart triggers in database

* Test for trigger restart on recovery

* Add order by clause and remove accidental overloading
2020-07-01 13:22:23 +00:00
Rohan Jacob-Rao
546c75b74f
Trigger service: Remove data in messages from TriggerRunnerImpl to Server (#6554)
* Remove data from message sent from TriggerRunnerImpl to Server

* Remove unused trigger name from config

changelog_begin
changelog_end
2020-06-30 21:02:10 -04:00
Rohan Jacob-Rao
ea16ff350d
Trigger service: Change to restart strategy with backoff (#6552)
* Add min/maxRestartInterval to configs and use in trigger restart strategy

* Adapt tests with triggers failing due to lack of network

changelog_begin
changelog_end

* Adapt tests for triggers with errors

* Remove comment about number of restarts

* Use a small initial restart interval for testing

* Remove old restart params

* Move maxInboundMessageSize to LedgerConfig

* Rename TriggerRunnerConfig to TriggerRestartConfig
2020-06-30 20:46:23 -04:00
nickchapman-da
353d0da6f7
Speedy ANF (#6440)
* ANF transformation in Speedy.

The idea behind this PR is to transform speedy expressions into a simpler form where all non-atomic sub-expressions are made explicit by the introduction of let-forms. In particular, for the function-application form. These simpler forms allow the execution engine to take advantage of the atomic assumption, and often removes many additional execution steps. In particular the pushing of continuations to allow execution to continue after a compound expression has been reduced to a value.

changelog_begin
changelog_end

* improve comment

* inline functions relocateA/L

* remove comment about scalafmt

* remove commented out alterative def for transformLet1

* improve code by adding incr methods to DepthA/E

* remove (n == 0) special case in trackBindings

* clarify comment further

* improve validate/go to not consume stack for deeply right-nested let-expressions

* address comments from Remy: be private; use final case case; etc

* rename to unsafeCompilationPipeline

* add back some trailing commas

* remove commented-out debug line

* improve comment

* remove dev/debug code in compilationPipeline

* remove commented out code in SEAppGeneral.execute

* undo unrelated code improvement in SValue.scala

* fix compile. object Anf cannot be private
2020-06-30 20:52:44 +00:00
Rohan Jacob-Rao
99a9662781
Trigger service: change convention for updating running trigger store (#6532)
* Don't modify running trigger store via the trigger runner actor

changelog_begin
changelog_end

* Remove runner actor from RunningTrigger type

And parent actor from TriggerRunnerImpl

* Add to running trigger store on start request

* Update tests for new trigger store convention

* Fix test with network outage
2020-06-30 12:00:45 -04:00
Rohan Jacob-Rao
9c4daad473
Trigger service nits (#6528)
- Move Stop Message back to original place with other Server state messages
- Other style nits

changelog_begin
changelog_end
2020-06-29 13:25:35 -04:00
Shayne Fletcher
44bb7f5233
Spawn a ref ledger auth instance from text fixture (#6502)
changelog_begin
changelog_end
2020-06-25 15:53:34 -04:00
Rohan Jacob-Rao
674fecf73f
Trigger service: split up ServiceMain into multiple files (#6498)
* Move top level case classes to package object

changelog_begin
changelog_end

* Move Server Message classes to separate file

* Server in Server.scala and ServiceMain in ServiceMain.scala

* Copyright headers
2020-06-25 16:01:35 +00:00
Rohan Jacob-Rao
eb861ffd4a
Trigger service: split up Server companion object (#6491)
The Server object and especially the apply method is extremely big and
hard to follow. There are a great many functions nested within the apply
method. This PR attempt to organize things, mostly by moving local
functions into methods of the Server class. I think this makes things
easier to follow, and I think it's more conventional.

Note that with this change I added some implicit parameters to the
Server constructor, which I think makes sense for these kinds of values
(contexts, sequencer factory, etc.) I also moved the Message trait to
the top level, but we will probably it its own file, following this PR.
There are a few other shallow tweaks that I made along the way.

Open to feedback on the high level structuring, as I'm not super
familiar with idiomatic design in Scala.

changelog_begin
changelog_end
2020-06-25 10:22:53 -04:00
Shayne Fletcher
2f539c477a
exe lookup in runfiles (#6489)
changelog_begin
changelog_end
2020-06-25 09:48:55 -04:00
Stephen Compall
a51d0db8ff
set scalac -Xsource:2.13 -Ypartial-unification globally (#6469)
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts

* add now-referenced scalaz-core where needed

* work around bad type signatures in scalatest Aggregating, Containing

* unused Any suppression

* work around bad partial-unification wrought by type alias

* remove unused Conversions import

- not required in 4f68cfc480 either, so unsure how it's survived this long

* work around Future.traverse; remove unused show import

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* remove unused bounds

* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed

* longer comment on what the options do

- suggested by @stefanobaghino-da; thanks

* forget Future.traverse, just use scalaz, it knows how to do this
2020-06-24 16:51:24 -04:00
Shayne Fletcher
afc573d422
step 0 integration of ref-ledger-authentication (#6484)
changelog_begin
changelog_end
2020-06-24 13:40:03 -04:00
Rohan Jacob-Rao
10208fd194
Trigger service: read packages from database on startup (#6470)
* Move database initialization to Server apply method
* Reorder dao/server creation
* Read packages from database on startup
* Test starting a trigger after a shutdown
changelog_begin
changelog_end
2020-06-24 11:18:16 -04:00
Rohan Jacob-Rao
b10bfa1ba7
Trigger service: Write packages to database if we have one (#6424)
This is needed to recover state after the service shuts down or crashes.
We add a method to the RunningTriggerDao to persistPackages. This only
does something in the case of a DbTriggerDao. In any case the Server
keeps a package map in memory as it's required to construct a trigger runner.
Uploads of existing packages is considered harmless.

changelog_begin
changelog_end
2020-06-19 13:49:49 -04:00
Rohan Jacob-Rao
78d598ecf3
Trigger service: Improve formation of SQL queries (#6422)
Avoid the use of Fragment.const which interprets raw strings as SQL
queries without any checks. Use the `sql` string interpolation which I
found out does the right thing with Strings and other simple types.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-19 09:32:27 -04:00
Remy
1b1b4eab2c
Speedy: clean machine builder name (#6427)
* Address comment martin made in #6368

* changelog

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-19 14:58:13 +02:00
Rohan Jacob-Rao
0ca1288d9d
Trigger service: Increase server binding timeout (#6421)
To avoid timeouts in CI. Also factor out the implicit val one level
so it is used by both tests and main method.

changelog_begin
changelog_end
2020-06-18 19:54:24 +00:00
Rohan Jacob-Rao
5934b22dda
Trigger service: Refactor Server.addDar to take encoded dar (#6395)
Trigger service: Refactor Server.addDar to take encoded dar

This is so we can write the encoded packages to the database if we have
one (without re-encoding them).

changelog_begin
changelog_end
2020-06-18 12:37:43 -04:00
Remy
5a9e7ebc7c
Speedy: refactor Machine builders (#6368)
In this PR we cleanup the constructor for the speedy Machine.

* We remove the `case`  keyword since `Machine` is a stateful class,
* We replace the pre-existing builders with
  + one generic builder `Machine.apply`,
  + scenario specific builder,

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-18 15:39:55 +02:00
Rohan Jacob-Rao
5ca65ad69a
Trigger service: Refactor testing with and without database (#6373)
* Trigger service: Refactor testing with and without database

Following the approach taken in the HTTP JSON API.
Define an AbstractTriggerServiceTest class with abstract jdbcConfig
member. This is instantiated with None and Some in the in-memory and
database test classes respectively. Tests for the individual modes can
be added in these concrete classes.

changelog_begin
changelog_end

* Move db init and destroy into beforeEach/afterEach to avoid concurrent tests clashing
2020-06-16 18:00:20 -04:00
Rohan Jacob-Rao
3cfed4d7a3
Trigger Service: Refactor running trigger access using DAO interface (#6334)
Trigger Service: Refactor running trigger access using DAO interface

This introduces a RunningTriggerDao trait and two classes that implement
it, one for in-memory and one for database storage. This moves a lot of
code out of the Server class and into the DAO classes. The class chosen
at runtime depends on whether a JDBC config has been provided.

The database DAO also has methods for initialization and clean up used by
the server main function and the test fixture.

This is a pure refactoring (modulo some minor error message changes) so
functionality should stay the same.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-13 14:33:46 -04:00
Shayne Fletcher
23e7dc826e
Trigger service docs and SDK integration basics (#6329)
* Integrate trigger-service into the assistant

changelog_begin
changelog_end

* Assistant integration and bare bones docs

changelog_begin
changelog_end
2020-06-12 18:54:38 +00:00
Simon Maxen
5329a4f97f
Ledger id requirement (#6323)
* Add option based constructor for LedgerIdRequirement

changelog_begin
changelog_end

* Make option based consructor the default, deprecate old constructor

* Update with review comments
2020-06-12 15:00:32 +00:00
Shayne Fletcher
0f47b8d1bf
Simplify findCredentials logic (#6301)
changelog_begin
changelog_end
2020-06-11 23:04:14 +00:00
Shayne Fletcher
b8126ee75c
Add --no-secret-key (#6309)
changelog_begin
changelog_end
2020-06-11 18:41:26 -04:00
Shayne Fletcher
07f24c0d93
Encrypt credentials (#6296)
* Encrypt credentials

changelog_begin
changelog_end

* Move key aquisition and write a big fat warning

* Only write encrypted token to DB.
2020-06-10 17:14:29 -04:00
Shayne Fletcher
033c58bc60
Switch to basic auth in readiness for auth service integration (#6279)
* Switch to basic auth in readiness for auth service integration

changelog_begin
changelog_end

* Newtype for user credentials
2020-06-10 09:48:33 -04:00
Rohan Jacob-Rao
e7017e6943
Trigger service: delete from database in stop trigger (#6249)
Includes a fair amount of refactoring to smooth the logic with/without a database. All tests now run with and without a database.

CHANGELOG_BEGIN
CHANGELOG_END
2020-06-08 16:19:35 -04:00
Shayne Fletcher
ecad4ff7c3
Fix test descriptions (#6269)
changelog_begin
changelog_end
2020-06-08 19:11:59 +00:00
Shayne Fletcher
0b7989a715
Adjust status code response for unauthorized (#6268)
changelog_begin
changelog_end
2020-06-08 18:56:43 +00:00
Shayne Fletcher
0625e68988
Disambiguate initialization halted vs. failed (#6248)
changelog_begin
changelog_end
2020-06-05 17:18:38 -04:00
Rohan Jacob-Rao
3159c617f6
Tweak error handling in listTriggers helper (#6247)
changelog_begin
changelog_end
2020-06-05 20:32:49 +00:00
Rohan Jacob-Rao
de2b5d195e
Trigger service style touch ups (#6246)
* Touch up logTriggerStatus

* Touch up removeRunningTrigger

* Touch up addRunningTrigger

* Remove IntelliJ (scalastyle I think) warnings about public members without type annotations

CHANGELOG_BEGIN
CHANGELOG_END

* Redundant brackets
2020-06-05 16:07:10 -04:00