Commit Graph

43 Commits

Author SHA1 Message Date
Andreas Herrmann
39a38d3a37
Update to Java 11 (#11512)
* Update to Java 11

changelog_begin
changelog_end

* Fix RoundingMode deprecation warnings

* Fix dep-ann warning

* Integer constructor

* JavaX annotation dependency

* javax.xml.bind was removed in Java 11

Using Guava as a replacement, since it is already a project dependency.

* JDK 11 no longer has a separate JRE tree

* Remove unused jdk_nix import

* remove now redundant jdk11_nix

* Java 8 --> 9 increased Instant.now() precision

See https://bugs.openjdk.java.net/browse/JDK-8068730

The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.

`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.

* TMP round timestamps

* Revert "TMP round timestamps"

This reverts commit af8e261278.

* Skip versions before 1.6.0 in migration tests

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-18 14:48:37 +00:00
Moritz Kiefer
c8ace8b1cf
Factor kind projector target into a variable (#11759)
Seems nicer than trying to format a string everywhere

changelog_begin
changelog_end
2021-11-18 09:51:43 +01:00
akshayshirahatti-da
e69a871e2f
[JSON-API] Configurable Hikari connection pool props (#11621)
* Changes to make certain hikari cp connection pool properties configurable via jdbc conf string

CHANGELOG_BEGIN
[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
CHANGELOG_END

* some missed changes for DbTriggerDao

* remove defaults for poolSize on JdbcConfig

* add constants for test defaults
2021-11-11 05:43:56 +00:00
akshayshirahatti-da
8d2b1b9ffe
[JSON-API] Port of http-perf-test to sandbox(aka next) (#11543)
* Changes to migrate http-perf-test back to sandbox with more parallelization for single user scenarios.
Increased parallelization is due to the architectural changes in sandbox where it uses
a tick every 100 millis to trigger stuff/data to be available on the read side

CHANGELOG_BEGIN
CHANGELOG_END

* Parallelization fixes for scenarios ExerciseCommand and SyncQueryNewAcs scenarios

* refactor sequential scenario run, make query part of SyncQueryVariableAcs run with single user
2021-11-09 08:44:37 +00:00
akshayshirahatti-da
55af7ad729
HttpServiceTestFixture provides a sandbox-classic ledger used for performance tests (#11128)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-05 21:10:09 +01:00
Stephen Compall
9fd8182bbb
move reusable functions from ContractsFetch to new fetch-contracts library (#11057)
* split akka-streams and doobie utils from ContractsFetch to new fetch-contracts library

* move most stream components from ContractsFetch to new library

* fix packages

* make the fetchcontracts domain model work

* move transactionFilter to fetchcontracts

* lots of unused imports

* start incorporating fetch-contracts in http-json

* move toTerminates

* more unused imports; http-json compiles

* more fetch-contracts dep

* bring back HasTemplateId[ActiveContract]; integration tests compile

* whole ledger-service tree compiles

* fix oracle missing dep

* scoping some new library identifiers

* remove apiIdentifier aliases

* comment on Aliases

* remove toTerminates shim

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* unused bazel imports

* remove already-addressed TODO

- suggested by @akshayshirahatti-da; thanks
2021-10-05 15:52:18 +00:00
akshayshirahatti-da
d3e6f16c61
[JSON-API] Migrating tests to use sandbox next (#11034)
* Changes to use sandbox next for our integration tests

CHANGELOG_BEGIN
CHANGELOG_END

* remove sandbox classic dependency for HttpServiceTestFixture and perf tests

* rely on sandbox next fixture test class

* add missing dependencies for http-json-oracle

* changes based on code review comments

* Add tag to skip test case for scala_2_12 and also add a jdbc backend for sandbox spun up for perf tests

* Reduce size of contracts for archiving test
2021-09-30 17:29:37 +01:00
pbatko-da
6dcdaa411c
[DPP-589] Add CLI flag to select minimum enabled TLS version (#10854)
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox

CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
2021-09-14 12:37:38 +02:00
akshayshirahatti-da
d809fd934a
[JSON-API] surrogate template id cache (#10806)
* Initial changes to add a surrogate_template_id cache to reduce db queries

CHANGELOG_BEGIN
CHANGELOG_END

* refactoring and addition of tests

* Code review based changes to use Contextual Logger and json-api metrics instance

* make max cache entries/size configurable

* Rename cache max entries default variable
2021-09-10 10:48:58 +00:00
Victor Peter Rouven Müller
5595d55c79
[JSON-API] Use the token from incoming requests to update the package list (#10602)
* Use the token from incoming requests to update the package list

changelog_begin
changelog_end

* Lazily initialize the ledger client

* Fix ee integration tests

* Fix package reloading behaviour by using a semaphore to check for ongoing updates

* Refactor out the semaphore code into a concurrency utility class

* Use correct locking for the updateTask so every thread always uses an up to date task

* Remove unused imports in utils.Concurrent & remove packages from the tests

* Hide & mark the token file cli option deprecated because we dont need it anymore and only keep it so client deployment code doesn't break

* Fix scala 2.12 build by adding more type annotations

* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/OptionParser.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Readd pgkManagementClient after it was removed accidentally (but now it's lazy)

* Remove concurrent object & use atomic boolean instead of a mutex because it makes more sense

* Replace semaphore with countdownlatch

* Refactor the caching into a separate class

* Use Instant instead of LocalDateTime

* Remove that ** of bad synchonization and do stupid simple synchronization because it JUST WORKS, besides adapt when we want to reload

* Remove await in tests because it can result in buggy tests

* remove unused code in WebSocketService.scala

* Unhide the access-token-file option as per request of Stefano

* Less implicit jwts per request of Stefano

* Try making some code more readable as by request of Akshay

* Use more shark because it expresses better than flatMaps if I don't need the arg

* Move defs in predicate in WebsocketService.scala around

* Try to minimize diff further in WebsocketService.scala

* Fix build and minimize diff in WebSocketService.scala further

* Minimize diff of function getTransactionSourceForParty in WebSocketService.scala

* Share the ec in WebSocketService.scala to minimize the diff

* Minimize in function predicate in WebSocketService.scala

* Further minimize in function predicate in WebSocketService.scala

* Change some case classes to be normal classes but with apply method

* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Get rid of implicit jwt tokens, the world is already confusing and full of implicits enough

* Improve readability

* Integrate the new LedgerClient which does not depend on a leder id

* Fix tests

* Apply suggestions from code review

thanks to @S11001001

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Apply further review comments

* Remove outcommented code

* Deprecate access token file option in the description too

changelog_begin

- [JSON API] The cli option `--access-token-file` is now deprecated. It
    is not needed anymore and you can safely remove it. Reason is that
    the operations which prior required a token at startup are now done
    on demand using the token of the incoming request.

changelog_end

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-09-03 10:12:01 +00:00
Stephen Compall
e5c4734780
enable JSON search index on Oracle by default (#10539)
* unconditionally enable JSON search index on Oracle

In '1kb of data' and larger Oracle integration tests:

ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3

From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB

    Cause: A query token is longer than 256 bytes

    Action: Rewrite query

* add changelog

CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
  ``--query-store-jdbc-config``, you must rebuild the database by adding
  ``,start-mode=create-only``.  See #10539.
CHANGELOG_END

* test only 1kb

* extra flag in db config string

* let Queries backends configure themselves from maps

* new Queries constructor dataflow to better support config values

* remove fields as we go, isolating backend-specific from -agnostic conf

- we use StateT to avoid the problems that will definitely arise if we
  don't DRY.

* fix up DbConfig including DbStartupMode

* start to uncouple json-api's config from db-utils

* two JdbcConfigs with different purposes/scopes

- also moves db-utils contents to com.daml.dbutils

* adapt trigger service to refactoring

* fix JdbcConfig leftovers

* adapt http-json-cli to new JdbcConfig

* remove extra ConfigCompanion

* explain more about the QueryBackend/Queries distinction

* split SupportedJdbcDriver into two phases with a tparam

* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass

* patch around all the moved objects with imports

* missed import from moving ConnectionPool to dbutils

* use new 2-phase SupportedJdbcDriver for ContractDao setup

* left off part of a comment

* more q.queries imports

* other imports from the dbutils move

* nested JdbcConfig

* configure the driver in each backend-specific test

* very confusing error, but make the imports nicer and it goes away

* nested JdbcConfig in perf

* missing newline

* port contractdao-bench

* test new option parsing all the way through QueryBackend

* disable search index for some tests, enable for others

* add changelog

CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
  options from JSON API.

- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
  index to speed up the queries endpoints.  However, Oracle versions
  prior to 19.12 have an unrecoverably buggy implementation of this
  index; in addition, the current implementation fails on queries with
  strings >256 bytes, with no way to disable the index for that query.

  Pass the ``disableContractPayloadIndexing=true`` option as part of
  ``--query-store-jdbc-config`` to disable this index when creating the
  schema.
  See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END

* port failure tests

* init version table last, drop first

- suggested by @realvictorprm; thanks

* rename split DBConfig.scala

- suggested by @realvictorprm; thanks

* move imports to not be in alphabetical order

- suggested by @realvictorprm; thanks

* remove createSchema

- suggested by @realvictorprm; thanks

* Revert "test only 1kb"

This reverts commit 616e173e63.

* port to scala 2.12

- bug in unused imports

- old name `-` for `removed`
2021-08-31 09:29:36 +00:00
Samir Talwar
4b8b67a1b5
Upgrade Scalatest to v3.2.9. (#10576)
* Upgrade Scalatest to v3.2.9.

Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.

The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.

As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.

CHANGELOG_BEGIN
CHANGELOG_END

* http-json-oracle: Fix a Scalatest dependency.

* ledger-api-client: Fix a Scalatest dependency.
2021-08-12 23:19:35 +00:00
akshayshirahatti-da
b00e146aa0
[JSON-API/trigger-service] Refactor db conn (#10497)
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils

* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end

* update trigger service to use new libs-scala/db-utils

* missed changes for http-json-oracle

* minor cleanup based on comments

* fix breaking scala 2_12 build

* cleanup db-utils/BAZEL.md file
2021-08-09 09:56:58 +00:00
Samir Talwar
df78f9cb1e
Replace LedgerConfiguration with InitialLedgerConfiguration or the load timeout. [KVL-1058] (#10487)
* participant-integration-api: Encapsulate the initial configuration.

* participant-integration-api: Reduce usage of `LedgerConfiguration`.

* Inline `LedgerConfiguration` wherever it's used.

Most things don't need all its constituent parts; this reduces the
amount of unused properties.

CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
  removed in favor of ``InitialLedgerConfiguration``. Its usage
  has been changed accordingly, with the ``configurationLoadTimeout``
  property becoming part of ``ApiServerConfig`` instead.

  The default options provided by ``LedgerConfiguration`` have been
  removed; you are now encouraged to come up with sensible values for
  your own ledger. The ``Configuration.reasonableInitialConfiguration``
  value may help.
CHANGELOG_END

* Correct the initial configuration submission delay for KV ledgers.

* kvutils: Mark supertype unused parameters as unused.

* kvutils: Extract out common configuration submission delays.

These values are specific to kvutils; other drivers should come up with
their own.

* configuration: Delete `NoGeneration`, as it's unused.
2021-08-05 16:31:45 +00:00
Victor Peter Rouven Müller
8603cd3621
[JSON-API] Move database independent tests into a seperate abstract test (#10482)
* [JSON-API] Move database independent tests into a seperate abstract test

The DatabaseStartupOps tests are now also tested against Oracle.
Besides, an additional test will now cover that table creation
doesn't run into name collisions for different table prefixes within
the same database.

changelog_begin
changelog_end

* Add missing copyright headers

* Adjusting the version query slightly to fix the oracle db integration tests

* Rewrite the version query of oracle to fix it (hopefully)

* Test the prefix collision the other way around

* Put the table prefix also infront of the ensure_json constraint in the oracle queries

* Convert the table name of the jsonApSchemaVersion table to uppercase so it can be found in the list of the created tables in Oracle.

* Fix scala 2.12 collection compatibility compiler error by using :+

* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Use flatTraverse instead of flatMap to fix the compile error in Queries.scala

* Process the startup mode also in the tests & error if it failed

* Add collections compat import to fix scala 2.12 build failure

* Be confused about the build error prior, revert the change

* Move dropAllTablesIfExist a bit down to have a better declaration order

* Extract the tables vector combined with the version table into a seperate val

* Remove debug in Queries.scala logging

* Make the initDatabaseDdlsAndVersionTable val lazy, so we don't get a nullpointer exception

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-08-05 16:47:19 +02:00
akshayshirahatti-da
4b55f1ab6a
Connection pool for Contract Dao (#10359)
* Switch ContractDao to use a HikariCP connection pool
CHANGELOG_BEGIN
CHANGELOG_END

* missed conn pool changes for PostgresTest and ContractDaoBenchmark

* shutdown db access await threadpool and fix formatting

* custom pool sizes for Prod and Integration similar to DbTriggerDao

* cleanup contract dao connection pool

* simplify Dao shutdown

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* remove redundant config setting

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* fix code formatting issue, NonUnitStatments warning
2021-07-27 09:15:39 +01:00
Moritz Kiefer
9c9b91eca7
Support deletion of a large number of contracts (#10353)
* Support deletion of a large number of contracts

fixes #10339

There are two orthogonal issues here:

1. scalaz’s toVector from the Foldable[Set] instance
   stackoverflows. I’ve just avoided using that altogether.
2. Oracle doesn’t like more than 1k items in the IN clause. I chunked
   the queries into chunks of size 1k to fix this.

changelog_begin

- [JSON API] Fix an error where transactions that delete a large
  number of contracts resulted in stackoverflows with the PostgreSQL
  backend and database errors with Oracle.

changelog_end

* fix benchmark

changelog_begin
changelog_end

* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Update ledger-service/db-backend/src/main/scala/com/digitalasset/http/dbbackend/Queries.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* that's not how you foldA

changelog_begin
changelog_end

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-07-22 06:47:26 +00:00
Samir Talwar
bd12170af2
participant-state: Remove the ParticipantId, PackageId, and Party aliases. [KVL-1002] (#10308)
* participant-state: Remove the `ParticipantId` alias.

This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.

CHANGELOG_BEGIN
CHANGELOG_END

* participant-state: Remove the `PackageId` and `Party` aliases.

They don't add anything. Let's just use `Ref`.

* kvutils: Restore missing compat imports.
2021-07-19 12:31:25 +00:00
Samir Talwar
6c63f96ba9
participant-integration-api: Move SeedService here. (#10261)
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.

CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
  *participant-state* Maven package to the *participant-integration-api*
  Maven package, under the Java package name
  ``com.daml.platform.apiserver`` to reflect its usage by the API
  server, not the participant state API. If you use this class directly,
  you will need to change your imports.
CHANGELOG_END
2021-07-14 08:35:29 +00:00
Remy
e4d7cd764a
LF: Deprecate com.daml.lf.data.TryOps.Bracket (#10249)
* LF: Deprecate com.daml.lf.data.TryOps.Bracket

CHANGELOG_BEGIN
CHANGELOG_END

* address reviews

* fix
2021-07-13 10:16:53 +00:00
Victor Peter Rouven Müller
4037b1cd9e
Add metrics to the http json service (#9923)
* Introduce metrics in the http-json service

changelog_begin

[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)

changelog_end

* Move MetricsReporter and it's dependencies into //ledger/metrics

* Restore non-ugly formatting for that one section in Endpoints.scala

* Update ledger/sandbox-common/src/test/lib/scala/platform/sandbox/cli/CommonCliSpecBase.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Hide metrics option for http-json

* Propagate context exception for the parseUri function in MetricsReporter.scala

* Move cliHint value above parseUri function to have a better structure (it's used once before it's defined and once after it's defined, which is weird to me)

* Use better value name than optMr & optFd in cliopts Metrics.scala

* Remove import order changes & whitespace changes

* Revert usage of Nanoseconds for conversion from scala duration to java duration to usage of Seconds

* Shorten hideIfRequested function

* Fix another rearranged import

* Fix another whitespace removal

* Readd metrics cli option to sandbox after refactoring

* Add missing return type annotation for invalidRead in MetricsReporter

* Readd newline in https OptionParser.scala

* Remove unecessary import

* Update ledger/sandbox-common/src/main/scala/platform/sandbox/cli/CommonCliBase.scala

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* Align setter & config name for metricsReportingInterval setting too in CommonCliBase.scala

* Rename http_json_api in Metrics.scala of metrics project to HttpJsonApi

* Reformat CommonCliBase.scala of sandbox-common project

* Fix CommonCliSpecBase test of sandbox

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-06-08 15:54:01 +00:00
Moritz Kiefer
25b7e544ce
Only resolve template id once (#9837)
While looking at errors produced during command submissions I got
confused by the fact that we resolve the template id in the command
service. Turns out there is no reason for doing that, our types are
just not precise enough. This PR fixes that by making sure that we
differentiate between commands where the template id has been resolved
and those where it hasn’t.

changelog_begin
changelog_end
2021-05-31 08:50:52 +02:00
Victor Peter Rouven Müller
beb2d138f2
Add cli option & system property to enable json only logging for http json service (#9725)
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
2021-05-20 13:28:34 +00:00
Victor Peter Rouven Müller
625592e7b5
Rename CorrelationID to InstanceUUID as it has been used wrongly. (#9672)
This ensures that we follow our internal guidance on logging correctly.

changelog_begin
changelog_end
2021-05-12 16:46:14 +00:00
Stephen Compall
ca9e89b3da
check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
* check whether collection.compat is unused when compiling for Scala 2.12

- Instead of always suppressing warnings for collection.compat._,
  we should only do it for Scala 2.13

- We can also reduce boilerplate by automatically adding this
  option when both silencer_plugin and collection-compat are
  present

CHANGELOG_BEGIN
CHANGELOG_END

* remove unused import

* remove another unused import

* remove even more unused imports

* missed compat dependency

* more missed compat dependencies

* missed compat dependency

* use scala_deps in scaladoc_jar

- #8423 inlined the major version expansion, but this seems to
  have been prior to proper support by scaladoc_jar

* restore custom handling of participant-integration-api

- fixing scaladoc_jar isn't worth it for a single case, as with
  deps vs scala_deps
2021-05-11 21:54:14 +00:00
Victor Peter Rouven Müller
4f0ceff7a2
Fix #7170 (#9618)
CHANGELOG_BEGIN

http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns

CHANGELOG_END
2021-05-11 14:15:17 +00:00
Stefano Baghino
43ffa426f3
Test for duplicate contracts when querying on behalf of multiple parties (#9443)
* Test for duplicate contracts when querying on behalf of multiple parties

Fixes #9388

changelog_begin
changelog_end

* Optimize imports

* Thanks to @S11001001 for answering the comment

* Re-structure the test following @S11001001's input in https://github.com/digital-asset/daml/pull/9443#discussion_r616083932 -- thanks
2021-04-20 09:25:40 +00:00
Stephen Compall
9ac74e697d
Oracle json-api: websockets and testing (#9278)
* comparison queries

* name the contract primary key constraint

* use ignore_row_on_dupkey_index instead of merge

- suggested by @cocreature in #9286 f7b2f14294fa33d6804251ce841529a1e2bd298d; thanks

* retrySqlStates for oracle

* enable all non-websocket tests

* name the template_id primary key constraint

* clean up concatFragment calls

* add Websocket tests for oracle

* move iouCreateCommand to be usable by oracle integration tests

* work around Scala 2.12 NPE in Source

* multiquery support for Oracle

* matchedQueries, therefore query-stream support for Oracle

* enable websocket tests

* test '& bar' and 5kb strings

- 5kb string fails on Oracle with
  ORA-01704: string literal too long

* refine the long data cases; gets too long at 4000 bytes as expected

- however, the predicate fails for unknown reason before then; possibly a missed
  escape character case

* handle long data with a fallback

- now the predicate fails in all cases instead of a SQL error, which is...better

* only interpolate true, false, null into JSON predicate conditions

- the problem was with JSON-formatted data; it must be SQL-typed instead

* adapt equal's large-data solution for comparison as well

- only works for numbers and strings, but that's all we need to compare

* move Implicits to Queries

* remove stray spaces in output

* test Oracle query expressions alongside Postgresql

* test that bools aren't compared like numbers and strings

* test @> conjunctions and special {}-query handling

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* note on PASSING ... AS X

- suggested by @cocreature; thanks

* remove printlns; these functions don't really need scaffolding anymore

- suggested by @stefanobaghino-da; thanks
2021-04-12 17:49:19 +00:00
Stefano Baghino
3fa05804bf
Make non-repudiation integration exclusive to enterprise edition (#9219)
* Make non-repudiation integration exclusive to enterprise edition

Closes #9182

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/9219#discussion_r599578614

* Address https://github.com/digital-asset/daml/pull/9219#pullrequestreview-618605511

* Add aliases for CE test targets, thanks to @S11001001 for the feedback

* Uh, turns out you need a main...

* Address https://github.com/digital-asset/daml/pull/9219#discussion_r599717526

* Add missing dependency for Oracle
2021-03-24 20:25:47 +01:00
Stefano Baghino
f0c8b1240d
Integrate non-repudiation into HTTP JSON API (#9180)
* Integrate non-repudiation into HTTP JSON API

changelog_begin
changelog_end

Closes #9094

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596400828

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596408663

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596402533

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596624708

* Address https://github.com/digital-asset/daml/pull/9180#discussion_r596624174

* Solve compilation error in daml-script

* Address https://github.com/digital-asset/daml/pull/9180#issuecomment-801875959

changelog_begin
- [JSON API] The JAR for the HTTP JSON API is no longer published on
Maven, use the fat JAR as indicated in the documentation.
changelog_end
2021-03-18 15:15:52 +00:00
Remy
a9b035e92e
LF: release LF 1.12 (#8856)
CHANGELOG_BEGIN

    - [LF] Release LF 1.12. This version reduce the size of transaction

    - [Compiler]: Change the default LF output from 1.8 to 1.11.

CHANGELOG_END
2021-02-24 21:17:33 +01:00
Moritz Kiefer
153a23d315
Port the rest of //ledger-service/... to 2.13 (#8836)
changelog_begin
changelog_end
2021-02-12 17:26:19 +01:00
Stephen Compall
00a5fe50ca
soft-code Postgres in json-api (#8517)
* SupportedJdbcDriver box for the required DB-specific implicits and magic values

* replace postgres references with the SupportedJdbcDriver box

* explaining the typeclass

* labels for debugging

* allow external initialization of SupportedJdbcDriver, but not usage

* thread SupportedJdbcDriver everywhere, hang it off of ContractDao

* remove unused dep from integration tests

* split Queries into an agnostic part and a DB-specific part

* document withOptPrefix

* reformat

* SQL syntax more amenable to refactoring

* different sets of DDL for different backends

* make everything use queries passed around everywhere (usually via ContractDao)

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END
2021-01-21 07:22:33 -05:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Moritz Kiefer
76b6fd86fb
Upgrade Scala dependencies for 2.13 compatibility (#8268)
* Upgrade Scala dependencies for 2.13 compatibility

This upgrades a bunch of Scala libraries to versions that have 2.13
support. There are two libraries that are still missing:

- diffson, this has a new version but with significant breaking
  changes and it is only used in Naigator console which I hope to kill
  before I have to worry about this.

- ai.x:diff, this is used in the ledger API test tool. The library is
  abondened but there are a few alternatives.

changelog_begin
changelog_end

* Fix pureconfig

changelog_begin
changelog_end

* Fix Navigator

changelog_begin
changelog_end
2020-12-14 09:59:00 +01:00
Moritz Kiefer
d83cbdb475
Upgrade akka and akka-http (#8048)
* Upgrade akka and akka-http

Was chasing an issue somewhere and thought this might affect it in
some way. It didn’t but I might as well turn the upgrade into a PR.

changelog_begin
changelog_end

* Fix trigger service tests

changelog_begin
changelog_end

* Downgrade akka-http again

changelog_begin
changelog_end

* Upgrade akka-http again and fix tests

changelog_begin
changelog_end

* Cleanup trigger service

changelog_begin
changelog_end
2020-11-25 10:13:51 +01:00
Moritz Kiefer
a74a1c15f4
Add healthcheck endpoints to JSON API (#7852)
* Add healthcheck endpoints to JSON API

This PR adds /livez and /readyz (following k8s naming scheme) that can
be used as liveness and readyness check. There isn’t much we can do
for liveness apart from showing that we can still respond to http
requests but readyness can be a bit more clever and check the ledger
connection as well as the database connection.

changelog_begin

- [JSON API] Add `/livez` and `/readyz` health check endpoints for
  easier integration with k8s and other schedulers.

changelog_end

* I hate windows

changelog_begin
changelog_end
2020-11-02 18:41:49 +01:00
Moritz Kiefer
339704dd14
Tests for timeouts to/from ledger in JSON API failure tests (#7791)
* Tests for timeouts to/from ledger in JSON API failure tests

changelog_begin
changelog_end

* Update ledger-service/http-json-testing/src/main/scala/com/daml/http/HttpServiceTestFixture.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Update ledger-service/http-json-testing/src/main/scala/com/daml/http/HttpServiceTestFixture.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* Apply suggestions from code review

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-10-26 09:08:03 +01:00
Moritz Kiefer
19a7f326f1
Include streaming API in JSON API failure tests (#7783)
This PR extends the failure tests with one that establishes a
websocket connection, kills the connection and reenables it at the
last offset.

changelog_begin
changelog_end
2020-10-23 09:12:09 +02:00
Moritz Kiefer
9da86a602e
Relax restrictions on JWTs on write endpoints in JSON API (#7712)
The previous restriction was both too lax and too strict now that we
have multi pary queries:

1. It allowed a party in `readAs` for command submissions which just
   fails on the ledger side. Changing this is technically breaking but
   only if you used a token that would have been rejected as soon as you
   enabled auth so that seems very resonable to break.

2. It didn’t allow extra parties in `readAs`.

This PR switches to requiring exactly one party in `actAs` while
supporting multiple parties in `readAs`.

changelog_begin

- [JSON API] JWTs on command submissions can now contain extra parties
  in the `readAs` field. `actAs` is still limited to a single party.

changelog_end
2020-10-20 19:37:44 +02:00
Moritz Kiefer
ea453c35cf
Add tests for connection failures in the JSON API (#7751)
* Add tests for connection failures in the JSON API

This PR adds some toxiproxy based tests to see how the JSON API reacts
if the connection to the ledger is killed. There are a bunch of
inconsistencies here in the tests some of which we might want to
address and the others we should at least document but I’ll leave that
for future PRs.

changelog_begin
changelog_end

* Import HttpServiceTestFixture instead of prefixing

changelog_begin
changelog_end
2020-10-20 18:59:11 +02:00
Leonid Shlyapnikov
20e2922b3e
JSON API perf test main that starts sandbox and json-api services (#7283)
* reintroducing the main

* Introducing `ledger-service/http-json-testing`

* cleaning up

* Starting sandbox and json-api from perf-test main

changelog_begin
changelog_end
2020-08-31 22:55:01 +00:00