Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.
[Here][1] is s a blog post by the Scala team about it.
Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.
Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.
changelog_begin
changelog_end
[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
* Change heartBeatPer to more intuitive naming of heartbeatPeriod
CHANGELOG_BEGIN
CHANGELOG_END
* Initial changes to add HOCON config for json_api
CHANGELOG_BEGIN
CHANGELOG_END
* avoid IllegalArgumentException noise
* use named arguments in big config conversion
* Changes include
- tests for a full http-json-api config file
- logging config and non-repudiation config is still specified via cli args.
- config readers for MetricsReporter
* Add defaults to WebsocketConfig case class to allow partially specifying fields on typeconf file
* changes to the JwtVerifierBase config reader and equivalent test
* message already describes the value
* replace manual succeed/fails with scalatest combinators
* use qualified imports for WebsocketConfig defaults
* add back autodeleted empty lines
* collapse two lists of token verifiers into one
* add new line to config files
* rename dbStartupMode to startMode to keep consistent with cli option and for easy documentation
* Changes to daml docs to specify ways to run JSON-API by supplying a HOCON config file.
CHANGELOG_BEGIN
JSON-API can now be started supplying a HOCON application config file using the `--config` option.
All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases.
CHANGELOG_END
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Changes to add the option of starting trigger service with typeconf/HOCON config
CHANGELOG_BEGIN
CHANGELOG_END
* add tests for authorization config and fail on both config file and cli args
* refactor and cleanup config loading and tests
* Changes based on code review comments
* Daml doc changes and making sure that we have defaults for most fields to mirror cli args
CHANGELOG_BEGIN
Trigger Service can now be configured with HOCON config file.
- If a config file is provided we will choose to start the service using that, else we will fallback to cli arguments.
- If both config file and cli args are provided we will error out.
CHANGELOG_END
* addressing some more code review comments
* use scalatest inside properly
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Changes to renable ws multiplexing
CHANGELOG_BEGIN
[TS-BINDINGS] Re-enable ws multiplexing for stream queries after resolving the reconnect connection close bug associated with ws state and liveness.
CHANGELOG_END
* websocket is passed as an argument to the onMessage handler
* consistently use 'manager' reference instead of 'this' in the handleQueries change method
* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Initial changes to add a surrogate_template_id cache to reduce db queries
CHANGELOG_BEGIN
CHANGELOG_END
* refactoring and addition of tests
* Code review based changes to use Contextual Logger and json-api metrics instance
* make max cache entries/size configurable
* Rename cache max entries default variable
* Use the token from incoming requests to update the package list
changelog_begin
changelog_end
* Lazily initialize the ledger client
* Fix ee integration tests
* Fix package reloading behaviour by using a semaphore to check for ongoing updates
* Refactor out the semaphore code into a concurrency utility class
* Use correct locking for the updateTask so every thread always uses an up to date task
* Remove unused imports in utils.Concurrent & remove packages from the tests
* Hide & mark the token file cli option deprecated because we dont need it anymore and only keep it so client deployment code doesn't break
* Fix scala 2.12 build by adding more type annotations
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/OptionParser.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Readd pgkManagementClient after it was removed accidentally (but now it's lazy)
* Remove concurrent object & use atomic boolean instead of a mutex because it makes more sense
* Replace semaphore with countdownlatch
* Refactor the caching into a separate class
* Use Instant instead of LocalDateTime
* Remove that ** of bad synchonization and do stupid simple synchronization because it JUST WORKS, besides adapt when we want to reload
* Remove await in tests because it can result in buggy tests
* remove unused code in WebSocketService.scala
* Unhide the access-token-file option as per request of Stefano
* Less implicit jwts per request of Stefano
* Try making some code more readable as by request of Akshay
* Use more shark because it expresses better than flatMaps if I don't need the arg
* Move defs in predicate in WebsocketService.scala around
* Try to minimize diff further in WebsocketService.scala
* Fix build and minimize diff in WebSocketService.scala further
* Minimize diff of function getTransactionSourceForParty in WebSocketService.scala
* Share the ec in WebSocketService.scala to minimize the diff
* Minimize in function predicate in WebSocketService.scala
* Further minimize in function predicate in WebSocketService.scala
* Change some case classes to be normal classes but with apply method
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Get rid of implicit jwt tokens, the world is already confusing and full of implicits enough
* Improve readability
* Integrate the new LedgerClient which does not depend on a leder id
* Fix tests
* Apply suggestions from code review
thanks to @S11001001
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Apply further review comments
* Remove outcommented code
* Deprecate access token file option in the description too
changelog_begin
- [JSON API] The cli option `--access-token-file` is now deprecated. It
is not needed anymore and you can safely remove it. Reason is that
the operations which prior required a token at startup are now done
on demand using the token of the incoming request.
changelog_end
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* unconditionally enable JSON search index on Oracle
In '1kb of data' and larger Oracle integration tests:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3
From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB
Cause: A query token is longer than 256 bytes
Action: Rewrite query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
CHANGELOG_END
* test only 1kb
* extra flag in db config string
* let Queries backends configure themselves from maps
* new Queries constructor dataflow to better support config values
* remove fields as we go, isolating backend-specific from -agnostic conf
- we use StateT to avoid the problems that will definitely arise if we
don't DRY.
* fix up DbConfig including DbStartupMode
* start to uncouple json-api's config from db-utils
* two JdbcConfigs with different purposes/scopes
- also moves db-utils contents to com.daml.dbutils
* adapt trigger service to refactoring
* fix JdbcConfig leftovers
* adapt http-json-cli to new JdbcConfig
* remove extra ConfigCompanion
* explain more about the QueryBackend/Queries distinction
* split SupportedJdbcDriver into two phases with a tparam
* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass
* patch around all the moved objects with imports
* missed import from moving ConnectionPool to dbutils
* use new 2-phase SupportedJdbcDriver for ContractDao setup
* left off part of a comment
* more q.queries imports
* other imports from the dbutils move
* nested JdbcConfig
* configure the driver in each backend-specific test
* very confusing error, but make the imports nicer and it goes away
* nested JdbcConfig in perf
* missing newline
* port contractdao-bench
* test new option parsing all the way through QueryBackend
* disable search index for some tests, enable for others
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END
* port failure tests
* init version table last, drop first
- suggested by @realvictorprm; thanks
* rename split DBConfig.scala
- suggested by @realvictorprm; thanks
* move imports to not be in alphabetical order
- suggested by @realvictorprm; thanks
* remove createSchema
- suggested by @realvictorprm; thanks
* Revert "test only 1kb"
This reverts commit 616e173e63.
* port to scala 2.12
- bug in unused imports
- old name `-` for `removed`
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils
* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end
* update trigger service to use new libs-scala/db-utils
* missed changes for http-json-oracle
* minor cleanup based on comments
* fix breaking scala 2_12 build
* cleanup db-utils/BAZEL.md file
* [JSON-API] Add option for setting a table prefix
changelog_begin
- [JSON-API] A table prefix can now be specified in the jdbc config via `tablePrefix=<YourFancyTablePrefix>`. This was added to allow running multiple instances of the json api without having collisions (independent of the chosen database).
changelog_end
* Extend the correct test in the oracle tests and simplify config override
* Fix formatting
* Fix postgres tests
* Fix bug in oracle query
* Fix typo
* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* add the table prefix to named constraints too
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* [JSON-API] Validate schema version & add minimal options for schema creation
* Add tests
* [JSON-API] Rework prior work and introduce the object SchemaHandling
* Add license headers & revert formatting changes
* Fix oracle build & scala 2_12 build
* correctly fix 2.12 build
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/SchemaHandlingResult.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* [JSON-API] Change case names & add backwards compat (but deprecate createSchema=true)
changelog_begin
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
changelog_end
* Add info about deprecated createSchema field
* Fix build & improve logging
* Give suggestions on what option to take, to fix an outdated or missing schema
* Renaming of schemaHandling to DbStartupMode, added more tests & correct exit codes depending on how the db startup went
* Align name with sandbox
* Improve tests
* Only add new sql code which strictly uses the interpolation to align with other pr's & minimally adjust statements
* Minimize diff
* Add backwards compat test
* Fix scala 2.12 build & oracle integration tests build
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Adjust code according to review request & tests & add a failure test
* If the call to initialize fails also log the error which was thrown
* Fix formatting
* Add missing collections compat import in integration tests
* Fix last build errors (scala 2.12) & use Either instead of Option for getDbVersionState
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
These were originally hidden in the first PR because the metrics were
very shaky. Now they are actually useful and mentioned in release
notes so hiding this option makes no sense.
changelog_begin
changelog_end
This solves two warts in the code:
- the validate/createUnsafe double-parse, because scopt doesn't let you flatMap;
- the non-JdbcConfig sub-configs appeared to need to know the JDBC drivers when
they really don't, because of a quirk in the inherited implementation
That coherence of scopt.Read instances calls for all its dependencies to be
coherent leads us to treat supportedJdbcDriverNames as a nullary typeclass
instance. This is a nullary typeclass by the same justification as
SupportedJdbcDriver; see scaladoc there for more.
And we solve the latter problem by...adding a type parameter, how else.
CHANGELOG_BEGIN
CHANGELOG_END
* Introduce metrics in the http-json service
changelog_begin
[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)
changelog_end
* Move MetricsReporter and it's dependencies into //ledger/metrics
* Restore non-ugly formatting for that one section in Endpoints.scala
* Update ledger/sandbox-common/src/test/lib/scala/platform/sandbox/cli/CommonCliSpecBase.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Hide metrics option for http-json
* Propagate context exception for the parseUri function in MetricsReporter.scala
* Move cliHint value above parseUri function to have a better structure (it's used once before it's defined and once after it's defined, which is weird to me)
* Use better value name than optMr & optFd in cliopts Metrics.scala
* Remove import order changes & whitespace changes
* Revert usage of Nanoseconds for conversion from scala duration to java duration to usage of Seconds
* Shorten hideIfRequested function
* Fix another rearranged import
* Fix another whitespace removal
* Readd metrics cli option to sandbox after refactoring
* Add missing return type annotation for invalidRead in MetricsReporter
* Readd newline in https OptionParser.scala
* Remove unecessary import
* Update ledger/sandbox-common/src/main/scala/platform/sandbox/cli/CommonCliBase.scala
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Align setter & config name for metricsReportingInterval setting too in CommonCliBase.scala
* Rename http_json_api in Metrics.scala of metrics project to HttpJsonApi
* Reformat CommonCliBase.scala of sandbox-common project
* Fix CommonCliSpecBase test of sandbox
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
CHANGELOG_BEGIN
- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`
CHANGELOG_END
Co-authored-by: victor.mueller@digitalasset.com <mueller.vpr@gmail.com>
