* Update to rules_haskell v0.16
* Update comments re bazel patches
* clean up bazel overrides
* Upgrade to Bazel 5.2.0
* Remove '--distinct_host_configuration=false'
* Update buildifier to 6.3.2
* Suffix macos and ubuntu caches with yyyymm
* bump windows cache to v14
* [REVERTME] bump linux/macos/darwin timeout to 4h
* added new end point to refresh the cache
* formatting
* returning old logic
* added logic to update the cache with a specific offset provided in the body
* formatting
* addressed comments
* formatting
* formatting
* formatting
* Return unit instead of List for processing refresh
* last changes on logic
* formatting
* simplify conversion
* comments addressed
The tests are not comprehensive.
We demonstrate that if the Oracle payload index is on, the names of fields with type Int may not exceed 251 chars.
For other configurations, Int and Text fields can have a name with lengths of at least 512 chars.
Also updated the naming and explanation of the guards which disable some tests when using Oracle with JSON index.
* add -Xlint options requiring no changes
* add -Xlint:recurse-with-default
- very minor code changes
* factor http-json hj_scalacopts duplication
* use lf_scalacopts_stricter in libs-scala where NonUnitStatements was
* use hj_scalacopts in api-type-signature
* add nonlocal-return and nullary-unit to hj_scalacopts
* commented-out excluded options
* add unit-special globally
* check implicit-recursion for clients code
* declare the separate kind of duplication
CHANGELOG_BEGIN
CHANGELOG_END
* move completely independent tests
* move many tests to 'QueryStoreIndependent'
* fix metering-report test to work with user tokens
* rename query-store/auth-dependent tests
- to QueryStoreAndAuthDependentIntegrationTest
- from AbstractHttpServiceIntegrationTestTokenIndependent
* security evidence moved
* point to #14844 and #15040 for some TODOs
* duplicate the Oracle tests, disable large literals when indexing is on
* split test file for parallelization
CHANGELOG_BEGIN
CHANGELOG_END
* move interface directory to api-type-signature; change bazel target; move scala files
* rename iface package to typesig; rename EnvironmentInterface to EnvironmentSignature
* rename Interface to PackageSignature
* rename fromReaderInterfaces to fromPackageSignatures
* rename InterfaceReader to SignatureReader
* rename InterfaceReader to SignatureReader in deprecation note
* rename readInterface to readPackageSignature
* rename InterfaceReaderError to Error
* rename InterfaceReaderMain to SignatureReaderMain
* rename InterfaceType to PackageSignature.TypeDecl
* rename astInterfaces to interfaces
* rename astInterfaces to interfaces in PackageSignature
* rename findAstInterface to findInterface
* rename a couple arguments
CHANGELOG_BEGIN
- [Scala API] The "quasi-public" API in ``com.daml.lf.iface`` has moved
to the new ``com.daml.lf.typesig`` package, with many accompanying
name changes as documented in `issue #13669
<https://github.com/digital-asset/daml/issues/13669>`__.
Daml 2.5.0 or later will remove the old names, and also rename the
Maven artifact from ``daml-lf-interface`` to
``daml-lf-api-type-signature``. We recommend that users of this API
use the deprecation guidance in 2.4.0 to port their applications
before this happens.
CHANGELOG_END
- double the resolveTemplateId interface and use the "contract type" for
exercise-by-id only
- lookup for template ID only
- 400 + error message test
- move query test to also run on PG/oracle
CHANGELOG_BEGIN
CHANGELOG_END
This expands the domain of the exercise endpoint to accept interface IDs
instead of template IDs when exercising by contract ID. The argument is
still [compatibly] called templateId; interface IDs are accepted nowhere
else.
CHANGELOG_BEGIN
- [JSON API] Supports interface choices, including exercise by contract
ID accompanied by an interface ID.
See `issue #13301 <https://github.com/digital-asset/daml/pull/13301>`__.
CHANGELOG_END
changelog_begin
[Trigger Service] Spurious logging warnings will not be printed when running `daml trigger-service`
[Oauth2 Middleware] Spurious logging warnings will not be printed when running `daml oauth2-middleware`
changelog_end
Removed references to a missing `stderr-appender` in several configuration file.
This removes warnings when running `daml trigger-service` and `daml oauth2-middleware`.
This will also remove those same warnings from several tests (HTTP JSON API and Trigger Service).
* WIP
* First version that compiles (with debug print statements)
* Revert changes on the failure tests
* Make all tests pass with some major cleanup of the new code
* Remove debug prints & outcommented code
* Make decoding / encoding test also compatible with user token tests
changelog_begin
changelog_end
* Minimize diff
* Fix oracle tests
* leave note for followup
* Update ledger-service/http-json/src/itlib/scala/http/AbstractHttpServiceIntegrationTest.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* SandboxNextFixture replaced by Sandbox-on-X based SandboxFixture
changelog_begin
changelog_end
* Some fixed tests
* No direct dependencies on //ledger/sandbox:sandbox and //ledger/sandbox:sandbox-scala-tests-lib
* Fix after rebase
* Rename SandboxFixture and add a missing dep
* Generate valid party names if hint is empty
* Smaller maxInboundMessageSize
* Added test for empty display name
* SandboxServer is a ResourceOwner
* Uses execution context passed as an input for resource management
* Fixes flaky FlywayMigrations issue with null Thread.currentThread.currentClassLoader
* SandboxServer simplification returns Port instead of ApiServer
* Dedicated PMAllocateWithoutDisplayName for non-Canton ledgers
* Created since Canton does not return empty display names
* Fixes OracleAround so it creates unique oracle users
* Fixes rouge connection pool in JdbcLedgerDaoTransactionsSpec
* Fixes cleanup in OracleAroundAll
* Introduces lockIdSeed for test frameworks
* Adapts usage
changelog_begin
changelog_end
* Change heartBeatPer to more intuitive naming of heartbeatPeriod
CHANGELOG_BEGIN
CHANGELOG_END
* Initial changes to add HOCON config for json_api
CHANGELOG_BEGIN
CHANGELOG_END
* avoid IllegalArgumentException noise
* use named arguments in big config conversion
* Changes include
- tests for a full http-json-api config file
- logging config and non-repudiation config is still specified via cli args.
- config readers for MetricsReporter
* Add defaults to WebsocketConfig case class to allow partially specifying fields on typeconf file
* changes to the JwtVerifierBase config reader and equivalent test
* message already describes the value
* replace manual succeed/fails with scalatest combinators
* use qualified imports for WebsocketConfig defaults
* add back autodeleted empty lines
* collapse two lists of token verifiers into one
* add new line to config files
* rename dbStartupMode to startMode to keep consistent with cli option and for easy documentation
* Changes to daml docs to specify ways to run JSON-API by supplying a HOCON config file.
CHANGELOG_BEGIN
JSON-API can now be started supplying a HOCON application config file using the `--config` option.
All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases.
CHANGELOG_END
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* make http-json:integration-tests into test suites
* make http-json-oracle:integration-tests into test suite
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove commented libraries
* Changes to make certain hikari cp connection pool properties configurable via jdbc conf string
CHANGELOG_BEGIN
[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
CHANGELOG_END
* some missed changes for DbTriggerDao
* remove defaults for poolSize on JdbcConfig
* add constants for test defaults
* split akka-streams and doobie utils from ContractsFetch to new fetch-contracts library
* move most stream components from ContractsFetch to new library
* fix packages
* make the fetchcontracts domain model work
* move transactionFilter to fetchcontracts
* lots of unused imports
* start incorporating fetch-contracts in http-json
* move toTerminates
* more unused imports; http-json compiles
* more fetch-contracts dep
* bring back HasTemplateId[ActiveContract]; integration tests compile
* whole ledger-service tree compiles
* fix oracle missing dep
* scoping some new library identifiers
* remove apiIdentifier aliases
* comment on Aliases
* remove toTerminates shim
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* unused bazel imports
* remove already-addressed TODO
- suggested by @akshayshirahatti-da; thanks
* Add failing test that covers the bug
* Fix on conflict error for inserts into the contracts table
changelog_begin
- [JSON-API] make key_hash indexes non-unique, this fixes a bug where a duplicate key conflict was raised on the query store when the same contract was being witnessed twice by two separate parties
changelog_end
* move test to parent so as to test oracle query store
* make key_hash indexes non-unique
* use recordFromFields
Co-authored-by: Akshay <akshay.shirahatti@digitalasset.com>
* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Changes to use sandbox next for our integration tests
CHANGELOG_BEGIN
CHANGELOG_END
* remove sandbox classic dependency for HttpServiceTestFixture and perf tests
* rely on sandbox next fixture test class
* add missing dependencies for http-json-oracle
* changes based on code review comments
* Add tag to skip test case for scala_2_12 and also add a jdbc backend for sandbox spun up for perf tests
* Reduce size of contracts for archiving test
* unconditionally enable JSON search index on Oracle
In '1kb of data' and larger Oracle integration tests:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3
From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB
Cause: A query token is longer than 256 bytes
Action: Rewrite query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
CHANGELOG_END
* test only 1kb
* extra flag in db config string
* let Queries backends configure themselves from maps
* new Queries constructor dataflow to better support config values
* remove fields as we go, isolating backend-specific from -agnostic conf
- we use StateT to avoid the problems that will definitely arise if we
don't DRY.
* fix up DbConfig including DbStartupMode
* start to uncouple json-api's config from db-utils
* two JdbcConfigs with different purposes/scopes
- also moves db-utils contents to com.daml.dbutils
* adapt trigger service to refactoring
* fix JdbcConfig leftovers
* adapt http-json-cli to new JdbcConfig
* remove extra ConfigCompanion
* explain more about the QueryBackend/Queries distinction
* split SupportedJdbcDriver into two phases with a tparam
* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass
* patch around all the moved objects with imports
* missed import from moving ConnectionPool to dbutils
* use new 2-phase SupportedJdbcDriver for ContractDao setup
* left off part of a comment
* more q.queries imports
* other imports from the dbutils move
* nested JdbcConfig
* configure the driver in each backend-specific test
* very confusing error, but make the imports nicer and it goes away
* nested JdbcConfig in perf
* missing newline
* port contractdao-bench
* test new option parsing all the way through QueryBackend
* disable search index for some tests, enable for others
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END
* port failure tests
* init version table last, drop first
- suggested by @realvictorprm; thanks
* rename split DBConfig.scala
- suggested by @realvictorprm; thanks
* move imports to not be in alphabetical order
- suggested by @realvictorprm; thanks
* remove createSchema
- suggested by @realvictorprm; thanks
* Revert "test only 1kb"
This reverts commit 616e173e63.
* port to scala 2.12
- bug in unused imports
- old name `-` for `removed`
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils
* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end
* update trigger service to use new libs-scala/db-utils
* missed changes for http-json-oracle
* minor cleanup based on comments
* fix breaking scala 2_12 build
* cleanup db-utils/BAZEL.md file
* [JSON-API] Move database independent tests into a seperate abstract test
The DatabaseStartupOps tests are now also tested against Oracle.
Besides, an additional test will now cover that table creation
doesn't run into name collisions for different table prefixes within
the same database.
changelog_begin
changelog_end
* Add missing copyright headers
* Adjusting the version query slightly to fix the oracle db integration tests
* Rewrite the version query of oracle to fix it (hopefully)
* Test the prefix collision the other way around
* Put the table prefix also infront of the ensure_json constraint in the oracle queries
* Convert the table name of the jsonApSchemaVersion table to uppercase so it can be found in the list of the created tables in Oracle.
* Fix scala 2.12 collection compatibility compiler error by using :+
* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Use flatTraverse instead of flatMap to fix the compile error in Queries.scala
* Process the startup mode also in the tests & error if it failed
* Add collections compat import to fix scala 2.12 build failure
* Be confused about the build error prior, revert the change
* Move dropAllTablesIfExist a bit down to have a better declaration order
* Extract the tables vector combined with the version table into a seperate val
* Remove debug in Queries.scala logging
* Make the initDatabaseDdlsAndVersionTable val lazy, so we don't get a nullpointer exception
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* [JSON-API] Add option for setting a table prefix
changelog_begin
- [JSON-API] A table prefix can now be specified in the jdbc config via `tablePrefix=<YourFancyTablePrefix>`. This was added to allow running multiple instances of the json api without having collisions (independent of the chosen database).
changelog_end
* Extend the correct test in the oracle tests and simplify config override
* Fix formatting
* Fix postgres tests
* Fix bug in oracle query
* Fix typo
* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* add the table prefix to named constraints too
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* [JSON-API] Validate schema version & add minimal options for schema creation
* Add tests
* [JSON-API] Rework prior work and introduce the object SchemaHandling
* Add license headers & revert formatting changes
* Fix oracle build & scala 2_12 build
* correctly fix 2.12 build
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/SchemaHandlingResult.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* [JSON-API] Change case names & add backwards compat (but deprecate createSchema=true)
changelog_begin
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
changelog_end
* Add info about deprecated createSchema field
* Fix build & improve logging
* Give suggestions on what option to take, to fix an outdated or missing schema
* Renaming of schemaHandling to DbStartupMode, added more tests & correct exit codes depending on how the db startup went
* Align name with sandbox
* Improve tests
* Only add new sql code which strictly uses the interpolation to align with other pr's & minimally adjust statements
* Minimize diff
* Add backwards compat test
* Fix scala 2.12 build & oracle integration tests build
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Adjust code according to review request & tests & add a failure test
* If the call to initialize fails also log the error which was thrown
* Fix formatting
* Add missing collections compat import in integration tests
* Fix last build errors (scala 2.12) & use Either instead of Option for getDbVersionState
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.
This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Remove an unused import.
* http-json-oracle: Remove `ledger-api-common` as a dependency.
* bindings-rxjava: Remove a now-unused dependency.
* participant-integration-api: Build Oracle tests, but don't run them.
CHANGELOG_BEGIN
CHANGELOG_END
* triggers: Switch to an environment variable for enabling Oracle tests.
* http-json: Switch to an environment variable for enabling Oracle tests.
* Disable running Oracle tests by default, not building them.
* triggers/service: Remove unused test dependencies.
* comparison queries
* name the contract primary key constraint
* use ignore_row_on_dupkey_index instead of merge
- suggested by @cocreature in #9286 f7b2f14294fa33d6804251ce841529a1e2bd298d; thanks
* retrySqlStates for oracle
* enable all non-websocket tests
* name the template_id primary key constraint
* clean up concatFragment calls
* add Websocket tests for oracle
* move iouCreateCommand to be usable by oracle integration tests
* work around Scala 2.12 NPE in Source
* multiquery support for Oracle
* matchedQueries, therefore query-stream support for Oracle
* enable websocket tests
* test '& bar' and 5kb strings
- 5kb string fails on Oracle with
ORA-01704: string literal too long
* refine the long data cases; gets too long at 4000 bytes as expected
- however, the predicate fails for unknown reason before then; possibly a missed
escape character case
* handle long data with a fallback
- now the predicate fails in all cases instead of a SQL error, which is...better
* only interpolate true, false, null into JSON predicate conditions
- the problem was with JSON-formatted data; it must be SQL-typed instead
* adapt equal's large-data solution for comparison as well
- only works for numbers and strings, but that's all we need to compare
* move Implicits to Queries
* remove stray spaces in output
* test Oracle query expressions alongside Postgresql
* test that bools aren't compared like numbers and strings
* test @> conjunctions and special {}-query handling
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* note on PASSING ... AS X
- suggested by @cocreature; thanks
* remove printlns; these functions don't really need scaffolding anymore
- suggested by @stefanobaghino-da; thanks
* Add Oracle support in the trigger service
This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.
changelog_begin
changelog_end
* use getOrElse
changelog_begin
changelog_end
