fixes#7097
changelog_begin
- [Triggers] The trigger service now has a `--port-file` option
matching the corresponding option in the JSON API.
changelog_end
Limiting this to a single one makes little sense and while you can
work around it by uploading more packages, that can be annoying during
development.
fixes#6332
changelog_begin
- [Triggers] The trigger service now accepts multiple `--dar`` options.
changelog_end
* Test all four trigger service configurations
- non-authenticated with in-memory backend
- non-authenticated with database backend
- authenticated with in-memory backend
- authenticated with database backend
changelog_begin
changelog_end
* add access token to trigger dao
* Simplify readRunningTriggers
* fmt
* Rename V2__Add_access_token.sha25 to V2__Add_access_token.sha256
* Apply suggestions from code review
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Separate trigger service test fixture
Define a dedicated fixture per resource instead of generating all
resources in a single fixture function. This allows to control the
lifetime of resources individually.
Reuse the same sandbox and toxiproxy and auth middleware for the test
suite instead of restarting new resources for each test case. This is to
reduce overall test runtime.
changelog_begin
changelog_end
* fix exec context
changelog_begin
changelog_end
* Use owned resources where appropriate
Avoid Await result and use owned resources instead.
* Reduce test timeout
* Be more patient for Windows
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Factor JWT verifier CLI flags
changelog_begin
changelog_end
* Use cli-opts in auth middleware
* Use cli-opts in sandbox cli
* Mark trigger service test as long
These have become prone to timeout on CI.
Increasing the size (timeout) is a temporary fix. A proper
solution is to a) not start a fresh sandbox per test-case and b)
separate the in-mem/db and no-auth/auth configrations into
separate Bazel test targets.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Configure a token verifier on the auth middleware
changelog_begin
changelog_end
* Verify the token in the auth middleware
changelog_begin
changelog_end
* Test token verification in auth middleware
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Requires authorization for `readAs:<party>` claims for the list and
status endpoints of the trigger service. In case of list the party is
provided by the request entity. However, in case of status the party is
determined by querying for the running trigger instance in the same way
as for the stop endpoint.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Authorize trigger service on middleware
changelog_begin
changelog_end
* Trigger service auth callback handler
* Forward token
* Do not pin the application ID in the access token
The trigger service will assign an individual application ID to each
trigger based on its UUID. Requiring tokens on the granularity of
application IDs would break the idea of storing the token in a cookie to
be able to use it across multiple requests.
changelog_begin
changelog_end
* todo persist trigger token
* Add a state parameter to middleware login
* add documentation comments
* typo
* fmt
* Align Party type between middleware and trigger service
The middleware was using `com.daml.lf.data.Ref.Party` while the trigger
service is using `com.daml.ledger.api.refinements.ApiTypes.Party` which
requires conversions. This aligns the types to avoid such conversions.
* optional application id in oauth2 test server
* align party types
* configure auth middleware in trigger service tests
* handle empty cookie header
* follow redirects in trigger service tests
* keep track of cookies
* keep track of cookies
* Replace any previous Cookie header
Otherwise on old daml-ledger-token cookie might persist and be preferred
over a newly added instance.
* DEBUG
* Configure test ledger client readAs claims
* fmt
* docstrings
* remove debug output
* Avoid endless redirect loops
When the replay still fails to authorize on the middleware then we do
not want to attempt another login flow.
* Store callback routes in authCallbacks
* fmt
* Push AuthTestConfig into test target
https://github.com/digital-asset/daml/pull/7654#discussion_r506510193
* Unbind oauth2 server after middleware
https://github.com/digital-asset/daml/pull/7654/files#r506513251
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* set doobie version to 0.9.2 and rerun maven pin
* port extractor and some of JSON API
* repin maven
* use doobie's own builder compatibility where required
* use probably bad derivations to supply Blockers where transactEC was required
- The point of using Blocker instead of ExecutionContext seems to be to
especially emphasize to API users that it isn't appropriate to use an
ExecutionContext with ordinary behavior. That is what we have done, which
should probably change, but just compiling for now.
* fix fragment inspection test for internal restructuring
- This test depends on implementation details of Doobie, so naturally it must be
altered when that runs. Fortunately, it's been made easier by the changes
in this upgrade.
* allow 256 blockers for navigator transaction blocker, like the global EC
* allow as many blockers as the pool size for trigger service
- The transactor shouldn't share ExecutionContext for transactions with the
caller, so we set up a new one based on configured pool size.
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* allow updating user state from high-level trigger rule
* fix tests and examples for new rule signature
* fix doc for new rule signature
* add changelog
CHANGELOG_BEGIN
- [Triggers] ``rule`` can now ``get`` and ``put`` the user-defined state just
like ``updateState`` does. It no longer accepts the state as an argument.
You can port your rule function types by replacing ``s -> TriggerA`` with
``TriggerA s``, removing the last argument from the function, and using
``get`` to retrieve the state at the beginning of the ``rule``'s ``do`` block,
if needed.
See `issue #7674 <https://github.com/digital-asset/daml/pull/7674>`__.
CHANGELOG_END
* test that rule userState changes propagate all the way out
* adapt compatibility filtering to #7681
* add ACS reader to TriggerA and TriggerStateA
* propagate changes in TriggerA, TriggerStateA structure
* allow query to be used in updateState and rule
* remove getTemplates
* remove ACS argument from updateState and rule
* fix type parameter order on query
* use query function in all tests and examples
* replace getContracts with query in documentation
* use wildcards instead of otherwise
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* rename TriggerStateA to TriggerUpdateA
- suggested by @cocreature; thanks
* missed renamings of TriggerStateA to TriggerUpdateA
- suggested by @cocreature; thanks
* make the meaning of the rlift functions clearer
* make initialize a TriggerInitializeA instead of a function; remove getContracts
* update tests and examples for new initialize signature
CHANGELOG_BEGIN
- [Triggers] Trigger ``updateState``, ``rule``, and ``initialize`` functions no
longer accept an ``ACS`` argument; instead, they must use the ``query`` action
to query the ACS, similar to the same function in DAML Script.
See `issue #7632 <https://github.com/digital-asset/daml/pull/7632>`__.
CHANGELOG_END
* disable older compatibility trigger builds
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* add ActionState to the standard library
* use 1 ActionState, 1 get, 1 put in low-level trigger library
* introduce TriggerStateA for updateState
* fix tests and examples for new updateState signature
CHANGELOG_BEGIN
- [Triggers] The ``updateState`` function now returns a ``TriggerStateA``. This
is an action like ``TriggerA``, but doesn't permit emitting commands. Instead
of taking the state as an argument and returning a new state, you can
manipulate the state with ``get``, ``put``, and ``modify``. Any existing
``updateState`` can be ported by replacing ``s -> expr`` in the lambda
expression with ``-> modify $ \s ->``, and then made to look nicer from there
as desired.
See `issue #7621 <https://github.com/digital-asset/daml/pull/7621>`__.
CHANGELOG_END
* some DAML docs for updateState and TriggerStateA
* add getCommandsInFlight as a TriggerA action
* immediately update commandsInFlight on emitCommands
- delay until the rule had returned made sense before submitCommands was launched
immediately; this also makes `getCommandsInFlight` more sensible
* remove commands-in-flight argument from high-level Trigger rule
CHANGELOG_BEGIN
- [Triggers] The "commands in flight" or ``Map CommandId [Command]`` argument has been
removed from high-level trigger ``rule`` functions; instead, the current
commands-in-flight can be retrieved with the new ``getCommandsInFlight`` function, which
can be done immediately at the beginning of the rule's ``do`` block to preserve exact
existing trigger behavior.
See `issue #7600 <https://github.com/digital-asset/daml/pull/7600>`__.
CHANGELOG_END
* remove commands-in-flight argument from trigger tests
* update doc examples and copy in compatibility
- compatibility change will most likely entail another flag day in compatibility tests,
replacing last_pre_7456_trigger_version
* update doc text for getCommandsInFlight
* test that getCommandsInFlight gets updated during the rule
* flag day for trigger compatibility tests
sdk-version: 1.7.0-snapshot.20201006.5358.0.0c1cadcf
File: src/CopyTrigger.daml
Hidden: no
Range: 55:11-55:19
Source: typecheck
Severity: DsError
Message:
src/CopyTrigger.daml:55:12: error:
• Couldn't match type ‘TriggerA ()’ with ‘() -> TriggerA ()’
Expected type: Party
-> DA.Next.Map.Map CommandId [Command] -> () -> TriggerA ()
Actual type: Party -> ACS -> () -> TriggerA ()
• In the ‘rule’ field of a record
In the expression:
Trigger
{initialize = \ _acs -> (), updateState = \ _acs _message () -> (),
rule = copyRule, registeredTemplates = AllInDar, heartbeat = None}
In an equation for ‘copyTrigger’:
copyTrigger
= Trigger
{initialize = \ _acs -> (), updateState = \ _acs _message () -> (),
rule = copyRule, registeredTemplates = AllInDar, heartbeat = None}
* match docs on TriggerAState to current usage of these fields
* remove emittedCommands, as commandsInFlight is now kept up-to-date
* zoomed from where?
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Make /login endpoint compatible to auth0
- Make the authorization and token endpoints configurable
- Use `application/x-www-form-urlencoded` as specified in
https://tools.ietf.org/html/rfc6749#section-4.1.3
- Check the status code of the token endpoint response
- Fix the type of the token's `expires_in` field
(`Int` instead of `String)
changelog_begin
changelog_end
* Add Auth0 testing instructions
* Use native application type on Auth0
* scope to claims mapping todo note
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* conservatively move daml-script, trigger SValue interpreters to common library
* introduce expect and JavaList pattern for converters
* clean up trigger Converter Command interpretation
* add Church Free monad
* add an action language for trigger updates
* add expectE to remove some of the joins
* convert more of the converters to expect
* tool for unrolling Free/Roll
* split handleStepResult up and clean up its pattern
* handleStepFreeResult to interpret TriggerF
* replace Free Church with Pure/Roll free from Script
* newtype for ActionTrigger
* replace update in low-level Trigger with Free TriggerF
* submit one Commands at a time
* boolean blindness strikes again
* log missed TriggerF steps
* comment actual Submit contents
* match #7501 fromPureSExpr sig change in 00b80b8ea3
* avoid using forwardPort in runTrigger
* push State back into DAML, so it can be excluded from the action list
* push Message back into DAML, unifying the action language for initialState and update
* bringing TriggerF into initial state
* really add TriggerF into initial state, with all ports, tested
* add ActionTrigger class, express initialState in its terms
* add all TriggerF actions to existing TriggerA
* Trigger.rule will no longer have Time argument
* rename getS, setS to get, put, matching C.M.T.State from transformers
* make high-level Rule evaluate to the underlying TriggerF sequence
* Assert's testRule doesn't have a transform yet
* move DamlTuple2 to common converter library
- suggested by @cocreature; thanks
* combine the two Frees, provide from Script
* remove time argument from integration tests
CHANGELOG_BEGIN
- [Triggers] The ``Time`` argument was removed from the trigger rule function; instead, it
can be fetched within the ``TriggerA`` ``do`` block by ``getTime``, as with ``Update``
and ``Scenario``. The ``LowLevel`` trigger interface has been redesigned; such triggers
need to be rewritten or ported to high-level triggers.
See `issue #7456 <https://github.com/digital-asset/daml/pull/7456>`_.
CHANGELOG_END
* add trigger rule simulator to support Assert module
* missed new Free module
- left in script per @cocreature
* remove retract as we ended up using foldFree for that purpose instead
- suggested by @cocreature; thanks
* throw ConverterException instead of RuntimeException
- suggested by @cocreature; thanks
* remove Time argument from coin-upgrade-trigger
* port trigger service tests
* port trigger scenario test
* put TriggerSetup and TriggerRule into LowLevel.Trigger instead of unboxed Free
- suggested by @cocreature; thanks
* remove Time argument from trigger compatibility test
* submit commands as soon as each `emitCommands` is sequenced
- we still collect a list, but only for tracking commandsInFlight
* filter out compatibility tests for triggers before now
* remove commented imports, libraries from new shared converter
* make the TriggerF interpreter tail-recursive
* remove unused compatibility trait
* add back new state logging
* remove refactoring comment
* rewrite some LowLevel initialStates in do
* hide Daml.Script.Free from docs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* remove forwardPortInitialState
- suggested by @cocreature; thanks
* manually port low-level updates
- suggested by @cocreature; thanks
* remove forwardPort
- suggested by @cocreature; thanks
* fail faster on unrecognized TriggerF
- suggested by @cocreature; thanks
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Add design doc for authentication in the trigger service
This is a draft of how I currently imagine authentication in the
trigger service to work. Since the authentication middleware has to be
pluggable in the end anyway we need public documentation in the end
anyway and I find this much easier to manage than a google doc outside
of the repo which just never sees updates.
changelog_begin
changelog_end
* Update triggers/service/authentication.md
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
This is clearly a breaking change but triggers are still alpha so we
can get away this (confirmed with Bernhard).
changelog_begin
- [DAML Trigger] Daml.Trigger.Assert now uses DAML Script instead of scenarios.
changelog_end
* get a LoggingContext into the TriggerRunnerImpl
* make some implicits more implicitly scoped and explicitly ascribed
* make some private/final markings
* most of JsonFormat[Identifier] is in companion
* experimental LoggingContext with phantom type
* ActorContext#log isn't really doing that much
* more details of LoggingContextOf
* make LoggingContextOf compile
* add trigger message logging, yet without context
* fix parent compile errors
* use Config as the phantom for its own logging extensions
* LocalDateTimeFormat cleanup
* switch TriggerRunner to contextual logging
* add trigger definition ID to logs
* log trigger-submitted commands, fix trigger test compile
* log trigger stopping and DAR uploads
* add context to PostStop/PreRestart logs
* add changelog
CHANGELOG_BEGIN
- [Triggers] More detailed logging of trigger actions and trigger service actions.
See `issue #7205 <https://github.com/digital-asset/daml/pull/7205>`_.
CHANGELOG_END
* missed copyright header
* switch to Unit, scala/bug#9240 fixed
* ledger-api-client: `maxInboundMessageSize` -> `maxInboundMetadataSize`.
CHANGELOG_BEGIN
- [Scala Bindings] Rename a field in the ``LedgerClientConfiguration``
to ``maxInboundMetadataSize``, to match the builder Netty channel
builder. It was incorrectly named ``maxInboundMessageSize``, which is
a different channel property that configures the maximum message size,
not the header size.
CHANGELOG_END
* ledger-api-client: Introduce a `maxInboundMessageSize` config property.
We use this a lot; easier if it's in the configuration.
CHANGELOG_BEGIN
- [Scala Bindings] Replace the
``LedgerClientConfiguration.maxInboundMessageSize`` property with a
new one that represents the maximum size of the response body.
CHANGELOG_END
* Add a dummy OAuth2 authorization server
This is intended for use in the trigger service integration tests not
as an artifact that we ship to users.
changelog_begin
changelog_end
* Address review feedback
changelog_begin
changelog_end
* Remove authentication service from trigger service
This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).
While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.
changelog_begin
- [Trigger Service] Parties are now specified in request bodies as
opposed to via HTTP Basic auth. This is done in preparation for
running the trigger service against authenticated ledgers.
changelog_end
* Fix integration tests
changelog_begin
changelog_end
* depend on flyway in trigger service; duplicate FlywayMigrations from sandbox
* bring in postgres/h2 detection, other data-source elements needed; use scalalogging
* use DataSource instead of DriverManager for DbTriggerDao, pass to flyway too
- initialization order isn't quite right yet
* hardcode postgres in triggers flyway path
* found an init order that doesn't fail in tests
* better pools for production, and don't forget to close them
* move most init SQL to first migration file
* make logHandler implicit again
- seems to have been an accidental implication of #6334
* move remaining create to V1__Init.sql
* use Prod poolSize
* prove that PostStop signals are never seen (in tests)
* move all the shutdown to Stop message
* remove dead code in DbFlywayMigrations
* always "initialize" database
CHANGELOG_BEGIN
- [Trigger Service] The database format has changed to allow migrations in
future releases. Databases are always initialized or migrated to the current
version on start, so use of ``--init-db`` is no longer required.
See `issue #7092 <https://github.com/digital-asset/daml/pull/7092>`__.
CHANGELOG_END
* it should always be obvious when there is no escape
* remove logs from experiments
* remove receiveSignal PostStop
* remove unused imports
* factor --address, --http-port, --port-file options from http-json to cli-opts
- enabling reuse in trigger service
* use cli-opts for address and http-port options in Trigger service
* mark ServiceConfig and some defaults private
* use --address option to set up server
* document Setter
* test --address option is parsed
* missing (c) headers
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] Accepts a new ``--address`` option to listen for HTTP connections on
interfaces other than localhost, such as ``0.0.0.0`` for all addresses.
See `issue #7090 <https://github.com/digital-asset/daml/pull/7090>`__.
CHANGELOG_END
* test only 'restart triggers with update errors' in CI, 100 times
CHANGELOG_BEGIN
CHANGELOG_END
* build fewer things and try 1000 runs
* test restart after shutdown, 200 times
* distrust the log
* report which withTriggerService is being used to provide log dividers
* log when TriggerStarting, TriggerStarted received in 'starting' state
- In mainline, we only add logs when we add triggers in the 'running'
stat in Server.apply. Let's see if we aren't sending these messages
to the `Server.start` actor when it is still in the 'starting' loop
* log same thing in starting and running states
* revert build experiment and extra logs
- keep the fixture location log, as it's a very useful divider when
parsing test results
* turn on NonUnitStatements in trigger runner, service main code
* add some friendly discards
* add a not-so-friendly discard
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* one more discard to clarify an Await.ready's type
* explain unbind issue
* reenable 'restart triggers after shutdown'
CHANGELOG_BEGIN
CHANGELOG_END
* wait for everything to shut down before completing a withTriggerService fixture
- similar to a change to HttpServiceFixture.withHttpService in #4593,
but without the suppression of shutdown errors
* label the WithDb tests
* in CI, test only 'recover packages after shutdown', 50 times
* experiment: Process#destroy appears to be async
* is it in the in-between period?
* partial -> total
* replace some booleans with assertions for better error reporting
* make triggerLog concurrent
* close channel and file in other error cases for port locking
- suggested by @leo-da; thanks
* use port locking instead of port 0 for trigger service fixtures
* destroy one service at a time
* missed continuation in build script
* use assertion language for "restart triggers with update errors"
* Revert "is it in the in-between period?"
This reverts commit 211ebfe9d2.
* use better assertion language for "restart triggers with update errors"
* restore full CI build
This code has a few problems: the auth service test mode is not enabled, the
admin ledger should be separate from main ledger if auth is enabled, and the
retry strategy not executed.
The auth service is not used in these tests yet and we have an example
of how to do it correctly in the auth service fixture (and auth service
client tests). We can try to reuse that fixture instead or copy the
code over when necessary.
changelog_begin
changelog_end
* Trigger service tests: Make sure toxiproxy server is running before connecting client
changelog_begin
changelog_end
* Undo infuriatingly wrong camel case
* triggers: Use `FreePort.find()`.
* ports: Move `LockedFreePort` from postgresql-testing for reuse.
* triggers: Use `LockedFreePort` to avoid race conditions.
* ports + triggers: Move common port testing into the ports library.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Pull out a new `Main` object that wraps sandbox-classic.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Fail if a JDBC URL is not provided or not for PostgreSQL.
* sandbox-classic: Rename the conformance test H2 database.
* daml-on-sql + sandbox-classic: Report configuration errors cleanly.
This means letting `ProgramResource` catch the errors, log, and exit.
* daml-on-sql: Change the name logged on startup.
* daml-on-sql: Change the default participant ID.
* sandbox-common: Give the ledger name its own tagged string type.
* sandbox-classic: Generate random ledger IDs using the ledger name.
* daml-on-sql: Remove the banner, replacing it with a blank line.
* daml-on-sql: Enable strong seeding by default.
And weak seeding in the conformance tests.
* sandbox-classic: Move the ledger name to a separate parameter.
It's not really configurable.
* sandbox-classic: Move LedgerName from sandbox-common.
* daml-on-sql: Remove "-participant" from the participant ID.
* daml-on-sql: Use `Name` where possible.
* daml-on-sql: Make the ledger ID mandatory.
* Revert "sandbox-classic: Move LedgerName from sandbox-common."
This reverts commit 0dad1584a7.
* daml-on-sql: Print "DAML-on-SQL" in the CLI help, not "Sandbox".
* daml-on-sql + sandbox + sandbox-classic: Split out custom CLI parsing. (#6846)
* participant-state: Simplify naming the seeding modes.
* Method for service account request
* Extend domain and factor out sa/secure
* Methods to list service accounts and poll for service account
changelog_begin
changelog_end
* Run auth client tests 20 times
* Revert "Run auth client tests 20 times"
This reverts commit 5264ce6f30.
* Assert creds list for new service account is empty
* Request credential method
* Method to get a new credential id
* Method for fetching credential
* Login method (for ledger access token)
* Make authorize style consistent with rest
* Remove redundant test (subset of second test)
* Run auth client tests 20 times
* Fail future for http error responses and refine retry strategy
* Scalafmt
* Revert "Run auth client tests 20 times"
This reverts commit 2568dc6b47.
* Test getting service account before request
* sandbox-common: Move the ledgerIdMode up in the configuration.
* sandbox-classic: Move the configuration from sandbox-common.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix miscellaneous warnings caught by IntelliJ IDEA.
* set many extra scalac -Xlint options for all Scala projects
CHANGELOG_BEGIN
CHANGELOG_END
* move NoCopy to its own file
package.scala:18: warning: it is not recommended to define classes/objects inside of package objects.
If possible, define trait NoCopy in package data instead.
trait NoCopy {
^
* move more traits, classes, and objects to proper packages
- note that `package` is itself a scoping construct, so if your reason
is the apparent aesthetic of placing a bunch of things in one `package
object`, that is easily remedied by deleting the `object` keyword
* fix some type-parameter-shadow warnings
- I'm generally in favor of sensible name-shadowing, following the
"deliberately hide variables that should not be accessed here" school
of thought. But I think type name shadowing isn't quite as valuable
and more likely to confuse than general variable shadowing, so have
experimentally linted it out.
Example warning:
EventsTableFlatEventsRangeQueries.scala:11: warning: type parameter
Offset defined in trait EventsTableFlatEventsRangeQueries shadows class
Offset defined in package v1. You may want to rename your type
parameter, or possibly remove it.
private[events] sealed trait EventsTableFlatEventsRangeQueries[Offset] {
^
* fix more package-object-classes warnings
* fix an inaccessible warning
ContractsService.scala:197: warning: method searchDb in class ContractsService references private class ContractsFetch.
Classes which cannot access ContractsFetch may be unable to override searchDb.
def searchDb(dao: dbbackend.ContractDao, fetch: ContractsFetch)(
^
* enable -Xlint:infer-any
- continuing the saga of #6116, #6132
* enable -explaintypes for more detailed type errors
* missed header for NoCopy; probably should have left it in the package file
* misspelling in comment
* revert -Xlint:doc-detached
- there are a lot of these fixes, and they are noisy, so shifting to a
separate PR
- thanks to @leo-da for pointing out
* WIP Call authorize before start trigger
* Intro auth service client
* Auth service client test setup
CHANGELOG_BEGIN
CHANGELOG_END
* scalafmt
* Use Spray JSON format and slightly better uri handling
* Get auth client test to work and fiddle with uris
* Remove changes to Server
* Add package for URI
* Format
* Copyrights
* Update sandbox packages
* Use default wallclock time for auth service sandbox
* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
* Add details about validity windows
* Reword description of auth service concepts
* Elaborate on auth flow dealing with token/credential expiry
CHANGELOG_BEGIN
CHANGELOG_END
The running trigger store is now written to synchronously by the server, so there's no need for retry logic.
Also assert equality of trigger ids rather than an arbitrary predicate, as it gives better error messages and we don't use other types of predicates for this function.
CHANGELOG_BEGIN
CHANGELOG_END
* Add optional trigger id to startTrigger
CHANGELOG_BEGIN
CHANGELOG_END
* Read running trigger table in DbTriggerDao
And get rid of traverse type application!
(since we have partial unification on)
* Restart triggers in database
* Test for trigger restart on recovery
* Add order by clause and remove accidental overloading
* Add min/maxRestartInterval to configs and use in trigger restart strategy
* Adapt tests with triggers failing due to lack of network
changelog_begin
changelog_end
* Adapt tests for triggers with errors
* Remove comment about number of restarts
* Use a small initial restart interval for testing
* Remove old restart params
* Move maxInboundMessageSize to LedgerConfig
* Rename TriggerRunnerConfig to TriggerRestartConfig
* Don't modify running trigger store via the trigger runner actor
changelog_begin
changelog_end
* Remove runner actor from RunningTrigger type
And parent actor from TriggerRunnerImpl
* Add to running trigger store on start request
* Update tests for new trigger store convention
* Fix test with network outage
* Move top level case classes to package object
changelog_begin
changelog_end
* Move Server Message classes to separate file
* Server in Server.scala and ServiceMain in ServiceMain.scala
* Copyright headers
The Server object and especially the apply method is extremely big and
hard to follow. There are a great many functions nested within the apply
method. This PR attempt to organize things, mostly by moving local
functions into methods of the Server class. I think this makes things
easier to follow, and I think it's more conventional.
Note that with this change I added some implicit parameters to the
Server constructor, which I think makes sense for these kinds of values
(contexts, sequencer factory, etc.) I also moved the Message trait to
the top level, but we will probably it its own file, following this PR.
There are a few other shallow tweaks that I made along the way.
Open to feedback on the high level structuring, as I'm not super
familiar with idiomatic design in Scala.
changelog_begin
changelog_end
* add -Xsource:2.13, -Ypartial-unification to common_scalacopts
* add now-referenced scalaz-core where needed
* work around bad type signatures in scalatest Aggregating, Containing
* unused Any suppression
* work around bad partial-unification wrought by type alias
* remove unused Conversions import
- not required in 4f68cfc480 either, so unsure how it's survived this long
* work around Future.traverse; remove unused show import
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused bounds
* remove -Ypartial-unification and -Xsource:2.13 where they were explicitly passed
* longer comment on what the options do
- suggested by @stefanobaghino-da; thanks
* forget Future.traverse, just use scalaz, it knows how to do this
* Move database initialization to Server apply method
* Reorder dao/server creation
* Read packages from database on startup
* Test starting a trigger after a shutdown
changelog_begin
changelog_end
This is needed to recover state after the service shuts down or crashes.
We add a method to the RunningTriggerDao to persistPackages. This only
does something in the case of a DbTriggerDao. In any case the Server
keeps a package map in memory as it's required to construct a trigger runner.
Uploads of existing packages is considered harmless.
changelog_begin
changelog_end
Avoid the use of Fragment.const which interprets raw strings as SQL
queries without any checks. Use the `sql` string interpolation which I
found out does the right thing with Strings and other simple types.
CHANGELOG_BEGIN
CHANGELOG_END
Trigger service: Refactor Server.addDar to take encoded dar
This is so we can write the encoded packages to the database if we have
one (without re-encoding them).
changelog_begin
changelog_end
* Trigger service: Refactor testing with and without database
Following the approach taken in the HTTP JSON API.
Define an AbstractTriggerServiceTest class with abstract jdbcConfig
member. This is instantiated with None and Some in the in-memory and
database test classes respectively. Tests for the individual modes can
be added in these concrete classes.
changelog_begin
changelog_end
* Move db init and destroy into beforeEach/afterEach to avoid concurrent tests clashing
Trigger Service: Refactor running trigger access using DAO interface
This introduces a RunningTriggerDao trait and two classes that implement
it, one for in-memory and one for database storage. This moves a lot of
code out of the Server class and into the DAO classes. The class chosen
at runtime depends on whether a JDBC config has been provided.
The database DAO also has methods for initialization and clean up used by
the server main function and the test fixture.
This is a pure refactoring (modulo some minor error message changes) so
functionality should stay the same.
CHANGELOG_BEGIN
CHANGELOG_END
* Add option based constructor for LedgerIdRequirement
changelog_begin
changelog_end
* Make option based consructor the default, deprecate old constructor
* Update with review comments
Includes a fair amount of refactoring to smooth the logic with/without a database. All tests now run with and without a database.
CHANGELOG_BEGIN
CHANGELOG_END
* Touch up logTriggerStatus
* Touch up removeRunningTrigger
* Touch up addRunningTrigger
* Remove IntelliJ (scalastyle I think) warnings about public members without type annotations
CHANGELOG_BEGIN
CHANGELOG_END
* Redundant brackets
* Stop running trigger under DB write failure
changelog_begin
changelog_end
* Another fix relating to initialization retries
changelog_begin
changelog_end
* Handle the error case directly
This adds a function withTriggerServiceAndDb which runs a test twice, once with and once without a database, and succeeds if both succeed. This will be useful for reusing test logic with both backends and making sure behaviour is consistent. I have used this function where possible, but it won't work for everything until stop is implemented on the DB side.
At the moment this new function squashes two tests into one making it hard to tell whether it failed with or without the database. In a future PR I will investigate using an abstract class to run the tests separately (hopefully with altered descriptions).
This feature required a few changes in the process, mainly:
- Use PostgresAroundAll to connect/disconnect to the database before and after all tests run
- Add a destroy method to the TriggerDao to reset the database between tests
- Use the TriggerDao in the withTriggerService functions to initialize / clean up the database at the start / end of each test
- Sort trigger instances from list using Scala's sort, not relying on Postgres' ordering of UUIDs. This also means we need to use UUIDs for trigger instances in the tests and sort nonempty vectors in expected results.
* Insert running trigger to DB when using one
If the DB write fails, the server sends itself a
TriggerInitializationFailure message so that the corresponding trigger
runner is stopped and the table is in sync with the actors.
We still need to retry writes here.
Includes basic test that runs the server with a JDBC config set and adds
a trigger, expecting a new entry to be added to the DB. However does not
check the running trigger table which we can do once reads are
implemented.
changelog_begin
changelog_end
* Await on future in test
* Update to new assertTriggerIds
* Apply scalafmt suggestions
* Create index on party token
* Read db in list command
* Update comment in test script
* Remove outdated comment
* Fix strings in insert and select
* Clean up test
* Add a second trigger in the db test
* Fix comment in test script
* Comment db tables
* Order trigger instances in list command
* Comment about TriggerDao execution context
* Store trigger history
changelog_begin
changelog_end
* Harvest trigger histories
changelog_begin
changelog_end
* Switch to Vector over List (and other bits and bobs)
* Use a better verb for updating trigger status method
* Add a comment
* Fix mangled comments
* Pass JdbcConfig object to TriggerDao apply
* No need to return TriggerDao from init db
* Refactor introducing RunningTrigger type
* Rename triggerId -> triggerInstance and triggerOrigId -> triggerName
Note this also changes the start request parameter name to triggerName.
However I have not yet renamed triggerId in the response messages. We
should probably make it triggerInstance there too but in a later PR.
changelog_begin
changelog_end
* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
* Don't update running triggers until we know the trigger is running
changelog_begin
changelog_end
* Don't update running triggers until we know the trigger is running
Minimal database initialization with schemas for running_triggers and dalfs tables. The user passes in the database URL, username and password in a config string argument (approach and code adapted from the JSON API).
In future the idea is to also create a "service" role with permissions to read and write to the new tables. Then the user can pass in the service role to connect to the database when running the service for real.
* Use triggerId field in trigger start response
* Use triggerId field for stop trigger result
* Fix indentation and make yields consistent
* Use pair constructor for JsObject instead of Map
* Use triggerIds field in list triggers response
changelog_begin
changelog_end
* Adapt ResponseFormat from JSON API
* Add some type annotations
* Use response format with status and errors/result fields
* Update and refactor tests
changelog_begin
changelog_end
* Endpoint to list all triggers (not yet by party)
* Clean up test code a little
changelog_begin
changelog_end
* Test for listing running triggers
* Respond with JSON list instead of random text
* List triggers by party
Pass party name in request body.
Store another map of party name to set of trigger ids.
Also store party names in the values of the original trigger id map, so
we can update the party map when stopping a trigger.
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
Contributes to #4194.
Closes#4231.
Closes#5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
command submission. These fields were previously deprecated following the introduction
of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
* Tighten result type
Command execution can't result in a sequencer error
* New helper method for extracting used contracts
* New error clause
* Add a DAO query for the maximum time of contracts
* Implement algorithm for finding ledger time
CHANGELOG_BEGIN
CHANGELOG_END
* fixup ledgerTimeHelper
* Use new ledger time algorithm
* Mark LET/MRT as deprecated
CHANGELOG_BEGIN
- [Ledger API] DAML ledgers have switched to a new ledger time model.
The ledger_effective_time and maximum_record_time fields of command submission are deprecated,
the ledger time of transactions is instead set automatically by the ledger API server.
Ledger time is no longer strictly monotonically increasing, but only follows causal monotonicity:
ledger time of transactions is greater than or equal to the ledger time of any used contract.
See `#4345 <https://github.com/digital-asset/daml/issues/4345>`__.
CHANGELOG_END
* Add ledger time skew check
* Remove command updater
LET/MRT are now deprecated, this class is now useless
* Remove old time model validator
* Switch to new time model check: kvutils
* Switch to new time model check: in-memory ledger
* Switch to new time model check: SqlLedger
* Use initial ledger config
* Ignore user provided LET
* Use TimeProvider in submission services
* Use deduplication_time in daml-script runner
- Also remove unnecessary command completion output of CommandTracker.
- Remove usage of maximum record time in CommandTracker.
* Use arbitrary default value for deduplication time
* Use built-in Instant ordering
* Remove obsolete test
* Remove obsolete test: CommandStaticTimeIT
* Refactor test: TransactionMRTCompliance
* Disable test: CommandTrackerFlow timeout
* thread maxDeduplicationTime through to CommandTracker
* Improve test
* Refactor command client configuration
* Deduplication time should always use UTC
* Add missing method in TimedIndexService after rebase
* Put more details into the deduplication error response.
* Use system time for command dedup submittedAt.
* Use explicit UTC time source in command validator
* Revert CommandTracker[Flow] to previous completion-recovering-behavior
* Adapt scala client command config to new config params
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Support uploading DARs to the trigger service
This PR adds a new `upload_dar` endpoint that accepts a DAR as a
multi-part form request and adds it to the list of compiled packages.
I’ve also made the DAR passed in on startup optional now given the new
endpoint.
There is no endpoint for deleting a DAR so far but there is none on
the ledger API either so I think this not particularly urgent.
changelog_begin
changelog_end
* Address review comments
Previously the http endpoint for starting a trigger would always
return immediately. Based on the recent refactorings, we now do the
non-IO trigger initialization synchronously and return a failed http
status code with an error message.
This also refactors the code to only have one (mutable) set of
compiled packages which is a prerequisite for dynamic package uploads.
changelog_begin
changelog_end
Previously parts of the initialization, in particular, the code for
finding the filter and the heartbeat were part of the Runner. This led
to an akward API and didn’t really make any sense.
Now all of this code is part of a pure `Trigger.fromIdentifier`
method and the runner only takes care of actually running the
ledger. This could also be useful for the trigger service where we
might want to synchronously call `getIdentifier` so users get some
indication if there request even points to a valid trigger
directly. However, this is not tackled by this PR.
changelog_begin
changelog_end
Previously the runner class was in a weird state where it was specific
to a DAR but not to an individual trigger. This meant that you had to
pass around a fair bit of state which got a bit awkward. This PR
addresses this by making the trigger class specific to the trigger.
It also now accepts `CompiledPackages` instead of a DAR which should
make it easier in the trigger service to support dynamic package
uploads.
changelog_begin
changelog_end
Previously we assumed that the module name was globally unique in the
DAR which is definitely not guaranteed. Now we instead detect the
package id of the trigger library based on the type of the trigger we
are running which doesn’t fall apart if there are multiple versions of
the trigger library.
I’ve also removed the check for the package id of the trigger library
since I’d like the trigger runner to be backwarts compatible from now on (we
didn’t break that in a while).
This is slightly ugly since the Runner class is currently not specific
to a single trigger but only the individual methods are aware of the
specific trigger identifier. I’ll refactor this in a separate PR.
changelog_begin
changelog_end
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Extract a committer from the uses of `SubmissionValidator`.
This makes the clock injectable too.
* kvutils: Provide logging contexts in the `Runner`.
* sandbox: Remove the `StaticAllowBackwards` time provider type.
It's not used anywhere.
* sandbox: Fix warnings in CliSpec.
* sandbox: Ensure that we cannot specify both static and wall-clock time.
* sandbox-next: Crash if wall clock time is not specified.
* sandbox-next: Document more known issues in the new Sandbox.
* sandbox: Add a Clock (and some tests) to TimeServiceBackend.
* sandbox-next: Support static time.
CHANGELOG_BEGIN
- [Sandbox Next] Re-establish static time mode.
CHANGELOG_END
* ledger-on-(memory|sql): Expect a `() => Instant`, not a `Clock`.
* sandbox: Move more resource acquisition into the `owner`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Reimplement SandboxClientResource as a resources.Resource.
* codegen: Use resources in TestUtil.
* sandbox: Manage PostgreSQL in tests with ResourceOwners.
* Add first prototype of triggers as a service (TaaS)
This is an extremely basic version of the trigger as a service thingy.
Right now, it supports spawning triggers and stopping them but nothing
else.
There is a very simple test to check that it’s not completely broken.
changelog_begin
changelog_end
* Apply suggestions from code review
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* remove debugging output
* remove leftover import
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
