Includes a fair amount of refactoring to smooth the logic with/without a database. All tests now run with and without a database.
CHANGELOG_BEGIN
CHANGELOG_END
* Touch up logTriggerStatus
* Touch up removeRunningTrigger
* Touch up addRunningTrigger
* Remove IntelliJ (scalastyle I think) warnings about public members without type annotations
CHANGELOG_BEGIN
CHANGELOG_END
* Redundant brackets
* Stop running trigger under DB write failure
changelog_begin
changelog_end
* Another fix relating to initialization retries
changelog_begin
changelog_end
* Handle the error case directly
This adds a function withTriggerServiceAndDb which runs a test twice, once with and once without a database, and succeeds if both succeed. This will be useful for reusing test logic with both backends and making sure behaviour is consistent. I have used this function where possible, but it won't work for everything until stop is implemented on the DB side.
At the moment this new function squashes two tests into one making it hard to tell whether it failed with or without the database. In a future PR I will investigate using an abstract class to run the tests separately (hopefully with altered descriptions).
This feature required a few changes in the process, mainly:
- Use PostgresAroundAll to connect/disconnect to the database before and after all tests run
- Add a destroy method to the TriggerDao to reset the database between tests
- Use the TriggerDao in the withTriggerService functions to initialize / clean up the database at the start / end of each test
- Sort trigger instances from list using Scala's sort, not relying on Postgres' ordering of UUIDs. This also means we need to use UUIDs for trigger instances in the tests and sort nonempty vectors in expected results.
* Insert running trigger to DB when using one
If the DB write fails, the server sends itself a
TriggerInitializationFailure message so that the corresponding trigger
runner is stopped and the table is in sync with the actors.
We still need to retry writes here.
Includes basic test that runs the server with a JDBC config set and adds
a trigger, expecting a new entry to be added to the DB. However does not
check the running trigger table which we can do once reads are
implemented.
changelog_begin
changelog_end
* Await on future in test
* Update to new assertTriggerIds
* Apply scalafmt suggestions
* Create index on party token
* Read db in list command
* Update comment in test script
* Remove outdated comment
* Fix strings in insert and select
* Clean up test
* Add a second trigger in the db test
* Fix comment in test script
* Comment db tables
* Order trigger instances in list command
* Comment about TriggerDao execution context
* Store trigger history
changelog_begin
changelog_end
* Harvest trigger histories
changelog_begin
changelog_end
* Switch to Vector over List (and other bits and bobs)
* Use a better verb for updating trigger status method
* Add a comment
* Fix mangled comments
* Pass JdbcConfig object to TriggerDao apply
* No need to return TriggerDao from init db
* Refactor introducing RunningTrigger type
* Rename triggerId -> triggerInstance and triggerOrigId -> triggerName
Note this also changes the start request parameter name to triggerName.
However I have not yet renamed triggerId in the response messages. We
should probably make it triggerInstance there too but in a later PR.
changelog_begin
changelog_end
* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
* Don't update running triggers until we know the trigger is running
changelog_begin
changelog_end
* Don't update running triggers until we know the trigger is running
Minimal database initialization with schemas for running_triggers and dalfs tables. The user passes in the database URL, username and password in a config string argument (approach and code adapted from the JSON API).
In future the idea is to also create a "service" role with permissions to read and write to the new tables. Then the user can pass in the service role to connect to the database when running the service for real.
* Implement a simple profiler for DAML scenarios
The profiler runs a single scenario and records timing information when
each function (and some other closures) are entered and left. The
resulting information can be visualized as a flamegraph using
[speedscope](https://www.speedscope.app/).
The profiler works by instrumenting the CEK machine at the heart of
DAML Engine. Unfortunetaly, this causes a very small overhead on
non-profiling runs too. However, in my benchmarks I could not measure
any significant impact on the overall runtime at all. More precisely,
the overhead is as follows:
Every closure now has an additional field called `label`. In
non-profiling runs this field is always set to `null`. This field needs
to be allocated, copied whenever we copy a closure and scanned during
garbage collection. Additionally, whenever we enter a closure, we check
this field and whenever it is _not_ `null`, i.e. never during
non-profiling runs, we record an "open event" and set up a hook for the
corresponding "close event". Thus, the additional cost during
non-profiling runs are a single pointer comparison and a jump beyond
the "then branch".
Since this is still very much in active development, there are no
documentation, other than an entry in a README, and no tests yet. They
will come before we promote this. However, the UX will look very
different then since we already have plans to significantly change it.
CHANGELOG_BEGIN
CHANGELOG_END
* Run scalafmt
* Make profiling argument to PureCompiledPackges optional
* Fix a bunch of tests
CHANGELOG_BEGIN
CHANGELOG_END
* scalafmt is so annoying
CHANGELOG_BEGIN
CHANGELOG_END
* Apply simple suggestions
CHANGELOG_BEGIN
CHANGELOG_END
* Simplify and clarify the public interface to Speedy.
- Remove `isFinal`. A client just uses `run()`.
- Remove `toSValue`. The value in available in `SResultFinalValue(v: SValue)`.
- A client never directly access the `.ctrl` (or `.returnValue`) components.
- A client may use `setExpressionToEvaluate(expr)` to evaluate a new expression on an existing machine.
changelog_begin
changelog_end
* remove while loop which executes just once
* avoid unnecessary mutation when running speedy
Remove the `Ctrl` trait and separate `Machine.ctrl: Ctrl` into `Machine.ctrl: SExpr` and `Machine.returnValue: SValue` instead. This allows for avoiding dynamic dispatch on `ctrl` and instead allows for checking a pointer for `null` to decide if we have an expression that needs further break-down or a return value ready to be passed to the next continuation.
To make this check really only a pointer comparison we also needed to remove the abomination of "fully applied partially applied primitives". In order to achieve this, we check whether a PAP will be fully applied afterward when applying the last argument.
On the `collect-authority` benchmark, this increases throughput by around 13%, on another more computation heave benchmark by about 21%.
`collect-authority` benchmark on `master`:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
112.361 ±(99.9%) 1.965 ms/op [Average]
(min, avg, max) = (107.047, 112.361, 120.745), stdev = 3.493
CI (99.9%): [110.396, 114.326] (assumes normal distribution)
```
`collect-authority` benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
98.196 ±(99.9%) 1.933 ms/op [Average]
(min, avg, max) = (91.580, 98.196, 105.478), stdev = 3.436
CI (99.9%): [96.263, 100.129] (assumes normal distribution)
```
computation heavy benchmark on master
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
44.030 ±(99.9%) 0.742 ms/op [Average]
(min, avg, max) = (42.124, 44.030, 46.781), stdev = 1.319
CI (99.9%): [43.289, 44.772] (assumes normal distribution)
```
computation heavy benchmark on this branch:
```
Result "com.daml.lf.speedy.perf.CollectAuthority.bench":
36.222 ±(99.9%) 0.580 ms/op [Average]
(min, avg, max) = (34.897, 36.222, 39.787), stdev = 1.031
CI (99.9%): [35.643, 36.802] (assumes normal distribution)
```
changelog_begin
changelog_end
* Use triggerId field in trigger start response
* Use triggerId field for stop trigger result
* Fix indentation and make yields consistent
* Use pair constructor for JsObject instead of Map
* Use triggerIds field in list triggers response
changelog_begin
changelog_end
* Adapt ResponseFormat from JSON API
* Add some type annotations
* Use response format with status and errors/result fields
* Update and refactor tests
changelog_begin
changelog_end
Speedy: run() dont step()
- Running the Speedy machine with `run()` instead of `step()`
- Remove: `SResultContinue`
- Add: `SResultFinalValue(_)`
We change the top level control of Speedy: from machine.step() to machine.run, with the control of stepping while the machine returns SResultContinue moved into speedy itself. (And so SResultContinue is removed in favour of SResultFinalValue.) The main advantage of this approach is that the tight while loop can be moved inside the exception handler, rather than having to wrap the handler every step.
changelog_begin
changelog_end
