Commit Graph

14 Commits

Author SHA1 Message Date
Marton Nagy
54339ada82
Safeguard Oracle CI tests with lockIdSeed [DPP-802] (#12573)
* Fixes OracleAround so it creates unique oracle users
* Fixes rouge connection pool in JdbcLedgerDaoTransactionsSpec
* Fixes cleanup in OracleAroundAll
* Introduces lockIdSeed for test frameworks
* Adapts usage

changelog_begin
changelog_end
2022-01-26 00:54:17 +01:00
Simon Maxen
cc19df44a8
Persist transaction metering info [DPP-814,DPP-821] (#12274)
* Add transaction metering information to batch and persist to database

changelog_begin
Transaction Metering now persisted to database
changelog_end
2022-01-25 18:29:49 +00:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Moritz Kiefer
fa7663148a
Drop 2.12 versioned_scala_deps (#11748)
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.

changelog_begin
changelog_end
2021-11-17 22:13:08 +00:00
Moritz Kiefer
3cc9de564f
Introduce dependency on Oracle image version (#10597)
We want to rerun tests when the image changes which this PR should
hopefully accomplish.

changelog_begin
changelog_end
2021-08-17 13:28:26 +00:00
akshayshirahatti-da
5f7a3691a5
[JSON-API] Perf gatling MultiUserQueryScenario (#10422)
* Gatling MultiUserQueryScenario initial version, 100k ACS , 100 users used for testing conn pool performance
    CHANGELOG_BEGIN
    CHANGELOG_END

* Configurable scenario parameters via env variables
2021-08-16 17:01:27 +00:00
Samir Talwar
4b8b67a1b5
Upgrade Scalatest to v3.2.9. (#10576)
* Upgrade Scalatest to v3.2.9.

Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.

The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.

As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.

CHANGELOG_BEGIN
CHANGELOG_END

* http-json-oracle: Fix a Scalatest dependency.

* ledger-api-client: Fix a Scalatest dependency.
2021-08-12 23:19:35 +00:00
Marton Nagy
c0a24fe9b6
HA PoC behind a feature flag [DPP-426] (#10227)
* Removing previous Async commit features

Previous async commit features had
- async commit configured by conifg-param
- special treatments to stil force sync commit for certain threadpools
- special treatment to stil force sync commit on transaction level for certain transactions.

This is a preparation step to clean the path for adding a new approach for async commit treatment:
- only session/connection level async configuration
- no transaction level special treatments
- only enable async commit for specific Connection pools (where it is needed / is safe)

* Add DataSourceStorageBackend

- to spawn DataSources in a controlled fashion
  these will be needed in upcoming commits for the HikariCP
- DataSources can have Connection init hooks defined with the help of the InitHookDataSourceProxy (this is needed for HA implementation)
- added DataSourceConfig to capture needed level of fine-tuning for DataSource creation

* Switches to DataSource wrapping in HikariCP instantiation
* Adds DBLockStorageBackend

- this is the abstraction and the implementation of database level locking
- with support for Oracle and Postgres

* Adds HaCoordinator and implementation
* Wiring of HaCoordinator in parallel indexer
* Adds feature flag

changelog_begin
changelog_end
2021-07-22 00:36:32 +02:00
Stephen Compall
3113702dc0
index json-api signatories and observers on Oracle (#9895)
* nvarchar2 keys are text-incompatible, but varchar2 keys are fine

* commit the ACS update before query

* add changelog

CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
  ``--query-store-jdbc-config``, you must rebuild the database by adding
  ``,createSchema=true``.  See #9895.
CHANGELOG_END

* expand the InitDdl set to include materialized views

* replace search index with a materialized view that expands the stakeholders

* allow materialized views to be created in Oracle testing

* join and query the contract_stakeholders table for party-set membership

- restoring a few elements removed by 3e6661128d (#9484)
2021-06-17 11:52:37 -04:00
Stephen Compall
4e712a01a0
add oracle option for json-api perf runner (#9492)
* add oracle option to http-json-perf-binary-ee

* add oracle path to perf Main's JDBC bracket

* adapt to availableJdbcDriverNames; missing deps

* add changelog

CHANGELOG_BEGIN
- [JSON-API Perf] ``--query-store-index=postgres`` must be passed
  to select PostgreSQL query store performance testing; ``true``
  and ``yes`` are no longer supported.
  See `issue #9492 <https://github.com/digital-asset/daml/pull/9492>`__.
CHANGELOG_END
2021-04-27 08:58:44 -04:00
Brian Healey
ca66061b28
Ledger Api Oracle support (#9277)
* WIP : first cut at changed schema files for oracle
Define Oracle as DbType and handle necessary case match switches for it
recomputed shas for oracle migration scripts
Oracle fixtures
get things compiling
Able to connect to Oracle
Working through getting schema definitions functional with Oracle
runnable schema definitions only for active tables on oracle
delete commented lines in schema scripts
use oracle enterprise
correct inadvertently changed postgres schemas
WIP - latest oracle-ificiation
passing upload packages spec
add additional test for package upload entry read
correct typo in oracle database spec name
use BLOB for parties ledger_offset
package_entries use hex version of offset for range queries
reformat and update shas for sql scripts
binary numeric implicit conversion for oracle
correct duplicate exception text for oracle
parties test passing on oracle
add additional column to hold hex offset for party_entries
party_entries working for all dbs
scalafmt
Configuration ledger_offset should be BLOB
update sha of oracle sql files
enable passing tests in order
remove misleading null comments
define additional custom VARRAY types
add participant-integration-api-oracle tests to linux-oracle job
Add TODO for places where we need to deal with separate implicit imports for Oracle vs Postgres/H2
oracle implicit conversions for custom arrays and other problematic types

Do not override default debug level for all tests in participant-integration-api

CHANGELOG_BEGIN
Ledger API and Indexer Oracle Support
CHANGELOG_END

passing TransactionWriterSpec
passing JdbcLedgerDaoCompletionsSpec JdbcLedgerDaoDivulgenceSpec
passing JdbcLedgerDaoContractsSpec
All Oracle tests passing apart from one post-commit validation test

* Remove JdbcLedgerDaoValidatedOracleSpec as this is only relevant for classic postgres-backed sandbox

* rebase to master -- offsets are now varchar2 rather than blob

* remove use of DBMS_LOB operations

* remove all greater than/less than variants for DBMS_LOB

* revert postgres files that need not be touched

* code review feedback : avoid code duplication

* avoid indirection in type names for oracle arrays

* code review: HexString implicit conversions are not needed

* code review: Oracle case is not yet implemented for appendonlydao

* code review: Oracle case is not yet implemented for appendonlydao (cleanup import)

* code review: revert files that should not be touched

* address code review feedback: db specific imports for command completion become part of queries

* code review: perform db-specific reserved word escape to avoid case match

* code review: remove all dbms_lob comparison operations

* use simpler insert into with ignore dupes hint for oracle

* code review: avoid db specific match case in events range, use db specific limitClause

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle

* code review: restore group by on binary and array fields for non-oracle dbs, honour the calculation of limit size from QueryParty.ByArith

* code review: honour the calculation of limit size from QueryParty.ByArith

* code review: drop user after oracle test

* code review: remove drop user as it throws errors due to dangling sessions

* code review: revert incorrectly changed postgres schema files

* code review: clean up TODOs

* Remove // before hostname for consistency with other oracle connection strings

* code review: unambiguously scope table column referenced in select and where queries

* code review: correct duplicate table alias
2021-04-26 18:34:17 -04:00
Moritz Kiefer
f7b2f14294
Add Oracle support in the trigger service (#9286)
* Add Oracle support in the trigger service

This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.

changelog_begin
changelog_end

* use getOrElse

changelog_begin
changelog_end
2021-03-31 18:39:08 +02:00
Gary Verhaegen
cbba75a2c0
oracle: switch to entreprise (#9254)
* oracle: switch to entreprise

CHANGELOG_BEGIN
CHANGELOG_END

* --network host

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-03-29 16:56:30 +02:00
Moritz Kiefer
a93f89b6f8
Factor out an Oracle test fixture (#9191)
* Factor out an Oracle test fixture

We need to add Oracle support to other components like the trigger
service as well. At that point, we cannot assume that things don’t
stomp on each other so this PR adds a fixture that generates a random
user (which also comes with its own schema, creating new databases is
a bit different in oracle) which we use testing.

changelog_begin
changelog_end

* Update libs-scala/oracle-testing/src/main/scala/testing/oracle/OracleAround.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Less top-level stuff in traits

changelog_begin
changelog_end

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-03-19 16:37:36 +01:00