changelog_begin
[Trigger Service] Spurious logging warnings will not be printed when running `daml trigger-service`
[Oauth2 Middleware] Spurious logging warnings will not be printed when running `daml oauth2-middleware`
changelog_end
Removed references to a missing `stderr-appender` in several configuration file.
This removes warnings when running `daml trigger-service` and `daml oauth2-middleware`.
This will also remove those same warnings from several tests (HTTP JSON API and Trigger Service).
Fixes#13042
changelog_begin
[Trigger Service] Debug logging is now hidden by default. See #13042
changelog_end
Summary of changes:
- remove unused `exports_files`
- refactor trigger-service-binary rules in a single comprehension
- add the `logback.xml` file in the JAR root
Tested manually: listed the contents of the output of `bazel build //triggers/service:trigger-service-binary-ce_deploy.jar` and `bazel build //triggers/service:trigger-service-binary-ee_deploy.jar` before and after the fix, observed that the only difference is the `logback.xml` file in the JAR root; added a debug logger call and ran both JARs before and after the fix, observed that after the fix the debug logging line is no longer printed.
Saves about 90 seconds in test wallclock time. Other tests considered
either ran too short to bother splitting, or were Oracle and thus won't
affect the main run at all.
CHANGELOG_BEGIN
CHANGELOG_END
In SandboxFixture when mixing in the auth middleware fixture, set
up the ledger client with a user token instead of a claims token
when not running in the claims-token-specific auth tests.
Fixes#12831.
* note about inClaims
* prevent Product, Serializable warts with AuthServiceJWTPayload
CHANGELOG_BEGIN
CHANGELOG_END
Duplicates the trigger service's auth tests for the two modes of
the oauth test server from #12929, then selects out the
claim-token-dependent tests as explained in
https://github.com/digital-asset/daml/issues/12831#issuecomment-1048176312
Part of #12831.
* also suppress claims-only tests in oracle-with-oauth2-test
* use inClaims for several tests
CHANGELOG_BEGIN
CHANGELOG_END
TestMiddlewareUserToken now uses user tokens from the oauth
server as well, courtesy #12929. TestMiddlewareClaimsToken is the
only middleware-only (non-Client) test that switches the oauth
test server back to producing claims tokens, and contains all the
tests that depend on claims token semantics. These tests are in a
sense exercising the oauth server rather than the middleware.
The token returned by the oauth server is irrelevant for the behavior these
tests are exercising, so they are merely ported to always run on user tokens.
- CallbackUriOverride
- LimitedCallbackStore
- ClientLimitedCallbackStore
- ClientNoRedirectToLogin
- ClientYesRedirectToLogin
- ClientAutoRedirectToLogin
CHANGELOG_BEGIN
CHANGELOG_END
Fixes#12881 by making the oauth2 test server invent user tokens
instead of claim tokens if you ask it to. As with the claim
tokens, these aren't "real" tokens backed by a participant
server, so only a subset of the tests makes sense, and we
abstract over the shared compatible tests accordingly.
CHANGELOG_BEGIN
CHANGELOG_END
* leave trigger service port for already-submitted issue #12831
* spin off another test idea, #12989
* SandboxNextFixture replaced by Sandbox-on-X based SandboxFixture
changelog_begin
changelog_end
* Some fixed tests
* No direct dependencies on //ledger/sandbox:sandbox and //ledger/sandbox:sandbox-scala-tests-lib
* Fix after rebase
* Rename SandboxFixture and add a missing dep
* Generate valid party names if hint is empty
* Smaller maxInboundMessageSize
* Added test for empty display name
* SandboxServer is a ResourceOwner
* Uses execution context passed as an input for resource management
* Fixes flaky FlywayMigrations issue with null Thread.currentThread.currentClassLoader
* SandboxServer simplification returns Port instead of ApiServer
* Dedicated PMAllocateWithoutDisplayName for non-Canton ledgers
* Created since Canton does not return empty display names
* operate tokenProvidesClaims on user management service output
* ignore everything but the applicationId claim for StandardJWTPayload
* let the client tester work with non-custom tokens
* test claims check on non-custom token
* add changelog
CHANGELOG_BEGIN
- [Auth Middleware] Supports standard auth tokens for participant user management.
See `issue #12563 <https://github.com/digital-asset/daml/pull/12563>`__.
CHANGELOG_END
* Fixes OracleAround so it creates unique oracle users
* Fixes rouge connection pool in JdbcLedgerDaoTransactionsSpec
* Fixes cleanup in OracleAroundAll
* Introduces lockIdSeed for test frameworks
* Adapts usage
changelog_begin
changelog_end
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.
[Here][1] is s a blog post by the Scala team about it.
Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.
Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.
changelog_begin
changelog_end
[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
* Drop support for no seeding in sandbox-classic
Sandbox classic is going away in SDK 2.0 and so is support for v0
contract ids which is the only thing no seeding was used for.
This has been cleared by product.
I’ll drop v0 contract ids completely in #12464 as a follow-up. Just
wanted to factor this out for ease of review.
changelog_begin
changelog_end
* .
* .
* Split channel configuration from LedgerClientConfiguration
Fixes#12391
The channel configuration now has to be provided separately from the
configuration specific to the ledger client. In this way we avoid
situations where the builder is provided with some configuration
that gets overridden.
changelog_begin
[Scala bindings] The channel configuration has been split from the
LedgerClientConfiguration class. Provide the gRPC channel specific
configuration separately or use a builder. The channel configuration
no longer overrides the builder.
changelog_end
* Fix compilation issues in //ledger-service/...
* Change heartBeatPer to more intuitive naming of heartbeatPeriod
CHANGELOG_BEGIN
CHANGELOG_END
* Initial changes to add HOCON config for json_api
CHANGELOG_BEGIN
CHANGELOG_END
* avoid IllegalArgumentException noise
* use named arguments in big config conversion
* Changes include
- tests for a full http-json-api config file
- logging config and non-repudiation config is still specified via cli args.
- config readers for MetricsReporter
* Add defaults to WebsocketConfig case class to allow partially specifying fields on typeconf file
* changes to the JwtVerifierBase config reader and equivalent test
* message already describes the value
* replace manual succeed/fails with scalatest combinators
* use qualified imports for WebsocketConfig defaults
* add back autodeleted empty lines
* collapse two lists of token verifiers into one
* add new line to config files
* rename dbStartupMode to startMode to keep consistent with cli option and for easy documentation
* Changes to daml docs to specify ways to run JSON-API by supplying a HOCON config file.
CHANGELOG_BEGIN
JSON-API can now be started supplying a HOCON application config file using the `--config` option.
All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases.
CHANGELOG_END
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Changes to add a pureconfig-util module with some shared config readers, and cleanup some code from oauth2-middleware hocon
CHANGELOG_BEGIN
CHANGELOG_END
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/CliSpec.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Changes to add the option of starting trigger service with typeconf/HOCON config
CHANGELOG_BEGIN
CHANGELOG_END
* add tests for authorization config and fail on both config file and cli args
* refactor and cleanup config loading and tests
* Changes based on code review comments
* Daml doc changes and making sure that we have defaults for most fields to mirror cli args
CHANGELOG_BEGIN
Trigger Service can now be configured with HOCON config file.
- If a config file is provided we will choose to start the service using that, else we will fallback to cli arguments.
- If both config file and cli args are provided we will error out.
CHANGELOG_END
* addressing some more code review comments
* use scalatest inside properly
Somewhat error-prone, so please review carefully.
Reasons we need this:
- Some file types are not properly handled by the script.
- The only exclusion mechanism we currently have (`NO_AUTO_COPYRIGHT`)
is overly coarse.
CHANGELOG_BEGIN
CHANGELOG_END
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Changes to introduce HOCON+pureconfig for oauth2-middleware
CHANGELOG_BEGIN
CHANGELOG_END
* remove Cli arg parsing code + cleanup based on code review
* addition of a minimal config and changes to README.md
* keep existing cli args, but load from config file if provided
* fix broken docs build
* make tests OS independent
* Fail/error on supplying both config file and cli opts for startup, address code review comments
* concurrent: Replace `DirectExecutionContextInternal` with `parasitic`.
* concurrent: Rename `DirectExecutionContext` `parasitic`.
* Use `ExecutionContext.parasitic` instead of `DirectExecutionContext`.
We no longer need the latter.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix formatting.
* Add support for read as in the trigger service
changelog_begin
- [Trigger-Service] a list of read_as parties can now be supplied to the playload for the v1/triggers endpoint (i.e. field name is "read_as").
changelog_end
* Fix scala 2.12 build
* Remove not-null constraint from the new read_as column to fix oracle tests
* Fix read queries to respect that the read_as column values can be NULL
* Don't set any default values for the read_as column, it being null is fine
* Add test that covers the readas feature in the trigger service
* Use only alice and public as parties in the new test, don't want to mix up things unnecessarily
* Fix failing auth test
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Update triggers/service/src/test/scala/com/digitalasset/daml/lf/engine/trigger/TriggerServiceTest.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Address review comments/Add the not null constraint to the read as column for postgres
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Changes to make certain hikari cp connection pool properties configurable via jdbc conf string
CHANGELOG_BEGIN
[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
CHANGELOG_END
* some missed changes for DbTriggerDao
* remove defaults for poolSize on JdbcConfig
* add constants for test defaults
* Migrate trigger service tests to sandbox from sandbox-classic
CHANGELOG_BEGIN
CHANGELOG_END
* remove config override for EngineMode.Dev based on code review
* Suport multi-party readAs in triggers
fixes#7640
This does not yet include the trigger service. We’ll tackle that separately.
changelog_begin
- [Daml Triggers] Triggers now support readAs parties. They can be
specified via `--ledger-readas a,b,c`. As part of this change
``testRule`` gained an extra argument to specify the `readAs`
parties. If you previously used
```
testRule trigger party acsBuilder commandsInFlight s
```
you now need to use
```
testRule trigger party [] acsBuilder commandsInFlight s
```
changelog_end
* Update triggers/tests/src/test/scala/com/digitalasset/daml/lf/engine/trigger/test/AbstractFuncTests.scala
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* trigger-service: dev-mode-unsafe flag
This adds a `--dev-mode-unsafe` flag analogous to the trigger service
analogous to its counterpart in the trigger runner.
CHANGELOG_BEGIN
CHANGELOG_END
* fix tests
* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
At the moment, the OAuth2 Middleware prints its config at startup.
Unfortunately that config includes a secret, which presumably we'd
prefer not to show.
CHANGELOG_BEGIN
- The OAuth2 Middleware now obfuscates its Client Secret when logging
its config.
CHANGELOG_END
Fixes#10957
changelog_begin
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
changelog_end
In many network setups, there will be a more direct route for the
trigger service to contact the auth middleware than going back through
the frontend's public IP address (and possibly thus also through
intermediaries like an nginx reverse proxy etc.). In _some_ network
setups, it may not even be possible for the trigger service to reach the
auth middleware through its externally-visible address.
This PR caters to these cases by allowing the trigger service to use two
separate addresses for the auth middleware, an internal one the trigger
service uses when it needs to talk to the auth middleware, and an
external one used in generating URLs for external clients.
This is backwards-compatible: if the old option is used, we simply use
the same value for both.
CHANGELOG_BEGIN
- The Trigger Service can now accept separate `--auth-internal` and
`--auth-external` CLI arguments, where `--auth-internal` is the
address used by the Trigger Service to reach the Auth Middleware
directly, and `--auth-external` is the address the Trigger Service uses
in generated URLs sent back to the client. The `--auth` option remains
and keeps working as before, setting both internal and external
addresses to the same given value.
CHANGELOG_END
It seems a bit silly that the auth middleware would refuse a token that
has an `actAs` when it needs a `readAs`.
Reference: https://docs.daml.com/app-dev/authorization.html#access-tokens-and-claims
CHANGELOG_BEGIN
- The auth middleware will now consider that actAs credentials are valid
for readAs requests.
CHANGELOG_END
The workaround for https://github.com/akka/akka/issues/29841 is no
longer required as that issue has been fixed upstream and we have
updated to a version that includes the fix.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Test case for LockedFreePort not colliding with port 0
changelog_begin
changelog_end
* Discover dynamic port range on Linux
* Random port generator outside ephemeral range
* remove dev comments
* Draw FreePort from outside the ephemeral port range
Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.
LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.
But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.
* Select dynamic port range based on OS
* Detect dynamic port range on MacOS and Windows
* Import sysctl from Nix on MacOS
changelog_begin
changelog_end
* Windows line separator
* FreePort helpers visibility
* Use more informative exception types
* Use a more light weight unit test
* Add comments
* Fix Windows
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add a comment to clarify the generated port range
* fmt
* unused import
* Split libs-scala/ports
Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
The claims check in the auth middleware was switched around: in effect,
it checked that we did not receive _more_ than we asked for, rather than
checking we receive _at least_ what we asked for.
Of course this would still not let anyone run any trigger without the
proper access token, but it would let people list running triggers and
request (or stop) trigger executions.
CHANGELOG_BEGIN
- Fix a bug in the auth middleware where insufficient credentials could
still give access to list of running triggers.
CHANGELOG_END
* Do not require redirect URI upfront
changelog_begin
changelog_end
This allows users to configure the redirect URI after the middleware
client binding occurred such that dynamic port selection can be taken
into account.
* Use port 0 in trigger service fixture
changelog_begin
changelog_end
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* unconditionally enable JSON search index on Oracle
In '1kb of data' and larger Oracle integration tests:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3
From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB
Cause: A query token is longer than 256 bytes
Action: Rewrite query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
CHANGELOG_END
* test only 1kb
* extra flag in db config string
* let Queries backends configure themselves from maps
* new Queries constructor dataflow to better support config values
* remove fields as we go, isolating backend-specific from -agnostic conf
- we use StateT to avoid the problems that will definitely arise if we
don't DRY.
* fix up DbConfig including DbStartupMode
* start to uncouple json-api's config from db-utils
* two JdbcConfigs with different purposes/scopes
- also moves db-utils contents to com.daml.dbutils
* adapt trigger service to refactoring
* fix JdbcConfig leftovers
* adapt http-json-cli to new JdbcConfig
* remove extra ConfigCompanion
* explain more about the QueryBackend/Queries distinction
* split SupportedJdbcDriver into two phases with a tparam
* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass
* patch around all the moved objects with imports
* missed import from moving ConnectionPool to dbutils
* use new 2-phase SupportedJdbcDriver for ContractDao setup
* left off part of a comment
* more q.queries imports
* other imports from the dbutils move
* nested JdbcConfig
* configure the driver in each backend-specific test
* very confusing error, but make the imports nicer and it goes away
* nested JdbcConfig in perf
* missing newline
* port contractdao-bench
* test new option parsing all the way through QueryBackend
* disable search index for some tests, enable for others
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END
* port failure tests
* init version table last, drop first
- suggested by @realvictorprm; thanks
* rename split DBConfig.scala
- suggested by @realvictorprm; thanks
* move imports to not be in alphabetical order
- suggested by @realvictorprm; thanks
* remove createSchema
- suggested by @realvictorprm; thanks
* Revert "test only 1kb"
This reverts commit 616e173e63.
* port to scala 2.12
- bug in unused imports
- old name `-` for `removed`
