CHANGELOG_BEGIN
- [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on
CHANGELOG_END
I’ve kept the infrastructure for versioned_scala_deps around because
I’m optimistic and hope that eventually we’ll do another Scala upgrade.
changelog_begin
changelog_end
* add actAs, readAs to `meta` for create, exercise, createAndExercise endpoints
* use meta actAs, readAs to control how contract IDs are looked up for exercise
* outdated comments on JwtWritePayload and JwtPayload
* toSet1 operator to clean up some NEL manipulation
* take optional readAs argument for query endpoint
* use readAs for query POST
* check whether readAs is safe in query endpoint
* missed CommandMeta args in tests
* FetchRequest, a domain model to layer on "fetch" endpoint's ContractLocator
- ContractLocator was overloaded as a domain request model *and* a component
of other domain request models; the addition of new arguments means it can
no longer exactly meet the former, and adding "readAs" to it would poison it
for the latter cases.
* take readAs argument from fetch endpoint
* add readAs security check from query to fetch
* move jwt parties functions to util
* testing the party-set/JWT functions
* missing headers
* caught boolean blindness in readAs security checks
* test that meta params are used for commands
* make resolveRefParties do a subset check, too
* Revert "make resolveRefParties do a subset check, too"
This reverts commit 40a66f102c.
* test that the readAs auth check actually applies
* test that command service uses meta readAs, actAs
* note on test coverage
* add changelog
CHANGELOG_BEGIN
- [JSON API] ``actAs`` and ``readAs`` may be specified for create, exercise,
create-and-exercise, non-WS fetch, and non-WS query.
See `issue #11454 <https://github.com/digital-asset/daml/pull/11454>`__.
CHANGELOG_END
* no saving mallocs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* untabify
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* move design comment to comment from function name
- suggested by @cocreature; thanks
* remove unneeded variable
* refactor single-key callers of requestJsonReader
- suggested by @cocreature; thanks
* build error in ce
* diagnose Windows failure
* add missed http-json-testing requirement
* use readers as fetch/query party-set name
- suggested by @cocreature and @realvictorprm, thanks
* extra import
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Changes to make certain hikari cp connection pool properties configurable via jdbc conf string
CHANGELOG_BEGIN
[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
CHANGELOG_END
* some missed changes for DbTriggerDao
* remove defaults for poolSize on JdbcConfig
* add constants for test defaults
* FoldableContravariant, a mapping for Foldable instances
* use FoldableContravariant to specialize several ImmArraySeq, NonEmpty methods
* folding specializations for ImmArray
* a few docs for FoldableContravariant
* specializations for FrontStack
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* add PartySet alias for db-backend
* add PartySet alias for fetch-contracts
* add PartySet alias for http-json
* deprecate old apply
* quick builder for NonEmpty collections
* replace PartySet in db-backend
* replace PartySet in fetch-contracts
* lar.Party is also domain.Party
* add incl1 operator
* replace PartySet in http-json
* port tests
* into with Scala 2.12 needs collection-compat
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* simplify a couple functions that don't need so much data transformation now
* clean up some OneAnds and HKTs
* deal with Scala 2.12 without having warning suppression
* better, more obscure choice for Scala 2.12
* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* enumerating out-of-sync offsets at the DB level
* cleanup in lastOffset
* write the latest-requested-or-read offset when catching up
- Writing only the latest-read, as before, would imply unsynced offsets
that are actually well-synced. This puts the DB in a more uniform
state, i.e. it should actually reflect the single value that the
fetchAndPersist loop tries to catch everything up to.
* detecting lagging offsets from the unsynced-offsets set
- Treating every unsynced offset as a lag would make us needlessly retry
perfectly synchronized query results.
* add Foldable1 derived from Foldable for NonEmpty
* nicer version of the unsynced function
* ConnectionIO scalaz monad
* rename Offset.ordering to `Offset ordering` so it can be imported verbatim
* finish aggregating in the lag-detector function, compiles
* port sjd
* XTag, a scalaz 7.3-derived tag to allow stacked tags
* make the complicated aggregation properly testable
* extra semantic corner cases I didn't think of
* tests for laggingOffsets
* a way to rerun queries if the laggingOffsets check reveals inconsistency
* if bookmark is ever different, we always have to rerun anyway
* boolean blindness
* incorporate laggingOffsets into fetchAndPersistBracket
* split fetchAndPersist from getTermination and clean up its arguments
* just compose functors
* add looping to fetchAndPersistBracket
* more mvo tests
* test unsyncedOffsets, too
* Lagginess collector
* supply more likely actual data with mvo tests; don't trust Java equals
* rework minimumViableOffsets to track sync states across template IDs
* extra note
* fix the tests to work against the stricter mvo
* move surrogatesToDomains call
* more tests for lagginess accumulator
* add changelog
CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a multi-template query backed by database
could have an ACS portion that doesn't match its transaction stream, if
updated concurrently. This conditions is now checked and accounted for.
See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
CHANGELOG_END
* port toSeq to Scala 2.12
* handle a corner case with offsets being too close to expected values
* didn't need XTag
* [Daml error codes API] Further implementations
* Implements ErrorCode.asGrpcError (and test)
* Error code logging now accepts correlation id and an extra context map
* Full error context is included into enriched logging context
CHANGELOG_BEGIN
CHANGELOG_END
* Fixed Scala 2.12 compilation issues
* Test case for LockedFreePort not colliding with port 0
changelog_begin
changelog_end
* Discover dynamic port range on Linux
* Random port generator outside ephemeral range
* remove dev comments
* Draw FreePort from outside the ephemeral port range
Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.
LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.
But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.
* Select dynamic port range based on OS
* Detect dynamic port range on MacOS and Windows
* Import sysctl from Nix on MacOS
changelog_begin
changelog_end
* Windows line separator
* FreePort helpers visibility
* Use more informative exception types
* Use a more light weight unit test
* Add comments
* Fix Windows
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add a comment to clarify the generated port range
* fmt
* unused import
* Split libs-scala/ports
Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* unconditionally enable JSON search index on Oracle
In '1kb of data' and larger Oracle integration tests:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3
From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB
Cause: A query token is longer than 256 bytes
Action: Rewrite query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
CHANGELOG_END
* test only 1kb
* extra flag in db config string
* let Queries backends configure themselves from maps
* new Queries constructor dataflow to better support config values
* remove fields as we go, isolating backend-specific from -agnostic conf
- we use StateT to avoid the problems that will definitely arise if we
don't DRY.
* fix up DbConfig including DbStartupMode
* start to uncouple json-api's config from db-utils
* two JdbcConfigs with different purposes/scopes
- also moves db-utils contents to com.daml.dbutils
* adapt trigger service to refactoring
* fix JdbcConfig leftovers
* adapt http-json-cli to new JdbcConfig
* remove extra ConfigCompanion
* explain more about the QueryBackend/Queries distinction
* split SupportedJdbcDriver into two phases with a tparam
* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass
* patch around all the moved objects with imports
* missed import from moving ConnectionPool to dbutils
* use new 2-phase SupportedJdbcDriver for ContractDao setup
* left off part of a comment
* more q.queries imports
* other imports from the dbutils move
* nested JdbcConfig
* configure the driver in each backend-specific test
* very confusing error, but make the imports nicer and it goes away
* nested JdbcConfig in perf
* missing newline
* port contractdao-bench
* test new option parsing all the way through QueryBackend
* disable search index for some tests, enable for others
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END
* port failure tests
* init version table last, drop first
- suggested by @realvictorprm; thanks
* rename split DBConfig.scala
- suggested by @realvictorprm; thanks
* move imports to not be in alphabetical order
- suggested by @realvictorprm; thanks
* remove createSchema
- suggested by @realvictorprm; thanks
* Revert "test only 1kb"
This reverts commit 616e173e63.
* port to scala 2.12
- bug in unused imports
- old name `-` for `removed`
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils
* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end
* update trigger service to use new libs-scala/db-utils
* missed changes for http-json-oracle
* minor cleanup based on comments
* fix breaking scala 2_12 build
* cleanup db-utils/BAZEL.md file
* participant-integration-api: Use `Scheduler` instead of `Materializer`.
Simpler API, and we can inject a test scheduler to make the tests more
reliable.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Use a test scheduler in tests.
This makes the tests faster and more reliable.
* participant-integration-api: Cancel timeout when the config is found.
* participant-integration-api: Fail properly if config lookup fails.
* participant-integration-api: Handle failures in provisioning config.
* participant-integration-api: Test shutting down the config provisioner.
* participant-integration-api: Use the scheduler cancellations.
More useful than a boolean.
* participant-integration-api: Handle submission ID generation failure.
Unfortunately this is untestable, because the only output is logging,
which we can't test right now.
* resources-akka: Add `ResourceOwner.forCancellable`.
* participant-integration-api: Simplify the config provisioner.
This makes it a set of functions; we don't need a class. There is no
exposed behavior.
* participant-integration-api: Use the services EC to initialize config.
* participant-integration-api: Add comments around `release`.
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Revert "participant-integration-api: Handle submission ID generation failure."
This reverts commit 72b13771a7.
* participant-integration-api: Factor out a `LedgerConfigProvider` class.
Again.
TBH, it is cleaner than multiple parameter methods.
Also adds more Scaladoc.
* resources-akka: `ResourceOwner.forCancellable` is now generic.
It returns whatever type the `acquire` function returns, to allow for
subtypes of `Cancellable`.
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* logging-entries: Make `LoggingEntries` a non-case class.
There's no reason for it to need `equals`, etc.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Convert commands into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This also adds a couple of missing items, and always logs `workflowId`.
* participant-state: Convert updates into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This changes some of the logging context structure, but otherwise
everything remains the same.
* Make sure Scaladoc is lined up for modified code.
* new projection for aggregated matched-queries
We can redo all the template-ID matches (and payload query matches, if
needed) in the SELECT projection clause to emit a list of matchedQueries
indices SQL-side.
CHANGELOG_BEGIN
CHANGELOG_END
* selectContractsMultiTemplate always returns one query
* factoring
* remove multiquery deduplication from ContractDao
* test simplest case of projectedIndex; remove uniqueSets tests
* remove uniqueSets
* add more test cases for the 3 main varieties of potential inputs
* remove uniqueSets tests that were commented for reference
* remove unneeded left-join
* scala 2.12 port
* port Map test order to 2.12
* use SortedMap so the Scala version tests are unified
- suggested by @cocreature; thanks
* Removing previous Async commit features
Previous async commit features had
- async commit configured by conifg-param
- special treatments to stil force sync commit for certain threadpools
- special treatment to stil force sync commit on transaction level for certain transactions.
This is a preparation step to clean the path for adding a new approach for async commit treatment:
- only session/connection level async configuration
- no transaction level special treatments
- only enable async commit for specific Connection pools (where it is needed / is safe)
* Add DataSourceStorageBackend
- to spawn DataSources in a controlled fashion
these will be needed in upcoming commits for the HikariCP
- DataSources can have Connection init hooks defined with the help of the InitHookDataSourceProxy (this is needed for HA implementation)
- added DataSourceConfig to capture needed level of fine-tuning for DataSource creation
* Switches to DataSource wrapping in HikariCP instantiation
* Adds DBLockStorageBackend
- this is the abstraction and the implementation of database level locking
- with support for Oracle and Postgres
* Adds HaCoordinator and implementation
* Wiring of HaCoordinator in parallel indexer
* Adds feature flag
changelog_begin
changelog_end
* [JSON-API] Log json request & response bodies in debug
This also readds logging of incoming requests and the responses which are being send out.
changelog_begin
- [JSON-API] Logging of the request and response bodies are now available for appropriate requests if the chosen log level is equal or lower than DEBUG. These can then be found in the logging context of the request begin & end log messages (The field names in the ctx are "request_body" and "response_body").
changelog_end
* Move the http request throughput marking to the right place including the logging of the processing time
* Ensure that the processing time measuring is implemented consistent
* daml-lf/data: Truncate party names in log output, on request.
The party name can grow quite long, so we offer ledger implementors the
opportunity to truncate it in structured log output.
Unfortunately, because we use Logback through the global
`LoggerFactory`, there is no place to inject logging configuration. This
means we also need to use global, mutable state to configure logging
output. I have added a `LoggingConfiguration` class+object in Daml-LF
Data, which may not be the best place, but I can't think of a better
one right now. I suggest we leave it there until it has reason to grow,
at which point we may want to move it.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `ToLoggingValue` mixin-able.
* participant-integration-api: Truncate parties in filters when logging.
* participant-integration-api: Cast to `Party` for logging.
Invalid input should not break the request at this point. No assertions.
* daml-lf/data: Move `Party to LoggingValue` to a new package.
This avoids the transitive dependency issue most of the time.
* daml-lf-data: Move the `Identifier` logging to another package.
Again, reduces the need for transitively depending on _logging-entries_.
* logging-entries: Split from contextualized-logging.
This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Fix dependencies for 2.12.
* logging-entries: Missed one more Scala 2.12 dependency.
* release: Publish logging-entries.
* participant-integration-api: Remove the subscription ID.
Pretty sure it's not used except for a single log line, which makes it
useless as a correlation ID.
If we want to correlate logs, let's add a correlation ID.
* participant-integration-api: Move transaction requests to trace logging.
Most of the useful information is already in the logging context. We
don't need to log the data structure too.
CHANGELOG_BEGIN
- [Ledger API Server] The amount of data logged in the API transaction
service has been reduced at INFO level. Please enable TRACE logging to
log the request data structures.
CHANGELOG_END
* participant-integration-api: Reorder methods in ApiTransactionService.
* participant-integration-api: Add the word "request" to some log lines.
* participant-integration-api: Add a logging prefix for string offsets.
* ledger-api-domain: `immutable.Set` -> `Set`.
It's an alias.
* participant-integration-api: Log transaction filters on subscription.
* participant-integration-api: Log transaction filters.
Just the parties isn't enough information.
* participant-integration-api: Log the entire transaction request.
Structured, because otherwise it's hard to throw things away later.
* contextualized-logging: Avoid `View` because it's not in Scala 2.12.
* contextualized-logging: Add tests for booleans.
* contextualized-logging: Avoid methods that accept views.
Scala 2.12 really doesn't like me when I do that.
* participant-integration-api: One more try at building with Scala 2.12.
* contextualized-logging: Automatically convert logging values to strings.
For now, this has almost the same behavior, but it allows us to
customize the output in the future.
The main change is that the log format has gone from:
context: {a=b, x=1, foo=bar}
to:
context: {a: "b", x: "1", foo: "bar"}
* contextualized-logging: Move `writeTo` inside `LoggingValue`.
* contextualized-logging: Allow for more than just strings.
`null`, numbers, and sequences are now correctly logged.
The log format has gone from:
context: {a: "b", x: "1", foo: "bar", parties: "[alice, bob]"}
to:
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_BEGIN
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_END
* contextualized-logging: Extract the string serializer.
* Use non-string logging where possible.
* contextualized-logging: Split logging values from serialization.
So that callers don't have to know about Jackson.
* contextualized-logging: `SeqView` is `Iterable`. Don't need both.
* contextualized-logging: Make `ToStringToLoggingValue` a `val`.
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* contextualized-logging: Add a transient dependency for 2.12 only.
This required more infrastructure than I thought it would.
* kvutils: Make it explicit that we're logging the hashes of archives.
The implicit was found to be a little confusing.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* nvarchar2 keys are text-incompatible, but varchar2 keys are fine
* commit the ACS update before query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,createSchema=true``. See #9895.
CHANGELOG_END
* expand the InitDdl set to include materialized views
* replace search index with a materialized view that expands the stakeholders
* allow materialized views to be created in Oracle testing
* join and query the contract_stakeholders table for party-set membership
- restoring a few elements removed by 3e6661128d (#9484)
* upgrade scalacheck to 1.14.3
* regenerate maven_install files
* some different names and implicits
* remove some fromTryCatchNonFatal
* more porting
* port fromTryCatchNonFatal to attempt
* factor the assertions in SignatureSpec to avoid \/
* deal with invariant \/
* make partial unification do what we want
* \/, parse*, and toNel
* many uses of the .right method
* a legitimate use of fromTryCatchThrowable
* rebuild maven pins
* further invariant \/
* OneAnd and Nel interface changes
* further Either games
* \/ and reformatting
* \/ in http-json
* \/ in http-json
* deprecations
* more invariance
* cleanup unused
* more invariance; http-json compiles
* final either follies
* small 2.12 extra incompatibility
* rebuild deps
* revisit a couple earlier fixes using nicer expressions I learned later
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* repin 2.12
* Log templateId & choice name (if present) on command submissions in the json API
changelog_begin
- [JSON API] The template id & choice name (if present) are now logged on command submissions in the Json API (at trace level)
changelog_end
* Move the template id & the choice into the logging context
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/CommandService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/CommandService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Fix compile error due to scala 2.12 collection differences
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Make use of nameOf for naming telemetry traces in participant-integration-api
changelog_begin
changelog_end
* Add copyright header to Telemetry.scala
* Add a simple nameof macro lib which has one function to return the fqn of the current method
* Make use of the new nameof lib within the repo to shorten the code further
* Remove old dependency on nameof macro
* Readd lines which have been accidentally removed
* Pretty print daml.yaml in Daml ledger export
changelog_begin
changelog_end
* Use dynamic port in example-export generation
* Add daml.yaml to Daml ledger export golden test
* Sort data-dependencies for reproducibility
* Replace hashes in data-dependencies with a placeholder
To avoid test failure when any of these change.
* normalize data-dependencies path in scala client
So that the bazel build action is more reproducible itself.
* Fix Scala 2.12 build
* Factor out deleteRecursively
changelog_begin
changelog_end
* Generate ledger export in temp-dir
Build actions are not sandboxed in Windows. The ledger export will also
created a directory `deps` to store all the DALFs in. On Windows this
directory may persist across Bazel builds and may cause errors due to
attempts to overwrite an existing directory.
To avoid these issues the ledger export is generated into a temporary
directory and only the wanted files are moved to the expected output
locations in the bazel execroot afterwards.
* Close source
* Use replace instead of replaceFirst
The latter matches regex pattern instead of just substring matching
which fails with Windows paths due to `\` being interpreted as an escape
character.
* Normalize Windows \ to Unix /
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
Ths is required to get a version of Gatling that supports Scala
2.13 (and only that because they do not cross build).
Unfortunately the upgrade is a bit more annoying than I was hoping for:
Our custom gatling utils rely on parsing the simulation log. This is
an internal file format with zero documentation or stabilityt
guarantees and as expected it has changed in incompatible ways during
the upgrade.
Rather than trying to reverse engineer and adapt to changes everytime
we upgrade, this PR switches us to a slightly more supported codepath
by parsing the `stats.json` and `assertions.json` produced by the
highcharts stuff. Afaict this is also what for example the Jenkins
integration relies on so while it’s not completely public API it seems
like the best option I could find.
There are a few pieces of information we can’t get out of those
files. Specifically:
1. maxUsers: we only ever need one users anyway so not really relevant.
2. start, duration, end: no idea why we would want those. we want per
request metrics not the total duration.
3. geometric mean: slightly annoying, but avg & stdev should be good
enough™.
4. The scenario name: Not really an issue but if it is, we can
disambiguate by changing request names.
changelog_begin
changelog_end
* Adds NeverEqualsOverride, makes DTOs case classes once again
* Remove unit-test (comes later), function separation (authors opinion: current level of abstraction is good enough) TODO
* Remove TODO regarding immutable deduplications: Not worth to invest more time and effort into deduplications since upcoming porject is targeting the same
* Keep TODO related to TransactionAccepted case: to guide later work with unit-testing
* Make DBDTOV1 sealed
* Add scaladoc to StorageBackend interface
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
changelog_begin
changelog_end
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
Some cached threadpools weren't given names, meaning at runtime there
are a bunch of pool-x-thread-y threads. This makes it hard to understand
which threads are being used for what.
The following pool names were introduced:
append-only indexer: input-mapping-pool, batching-pool
ProgramResource: program-resource-pool
kvutils PackageCommitter: package-preload-executor
CHANGELOG_BEGIN
CHANGELOG_END
* add oracle option to http-json-perf-binary-ee
* add oracle path to perf Main's JDBC bracket
* adapt to availableJdbcDriverNames; missing deps
* add changelog
CHANGELOG_BEGIN
- [JSON-API Perf] ``--query-store-index=postgres`` must be passed
to select PostgreSQL query store performance testing; ``true``
and ``yes`` are no longer supported.
See `issue #9492 <https://github.com/digital-asset/daml/pull/9492>`__.
CHANGELOG_END
* WIP : first cut at changed schema files for oracle
Define Oracle as DbType and handle necessary case match switches for it
recomputed shas for oracle migration scripts
Oracle fixtures
get things compiling
Able to connect to Oracle
Working through getting schema definitions functional with Oracle
runnable schema definitions only for active tables on oracle
delete commented lines in schema scripts
use oracle enterprise
correct inadvertently changed postgres schemas
WIP - latest oracle-ificiation
passing upload packages spec
add additional test for package upload entry read
correct typo in oracle database spec name
use BLOB for parties ledger_offset
package_entries use hex version of offset for range queries
reformat and update shas for sql scripts
binary numeric implicit conversion for oracle
correct duplicate exception text for oracle
parties test passing on oracle
add additional column to hold hex offset for party_entries
party_entries working for all dbs
scalafmt
Configuration ledger_offset should be BLOB
update sha of oracle sql files
enable passing tests in order
remove misleading null comments
define additional custom VARRAY types
add participant-integration-api-oracle tests to linux-oracle job
Add TODO for places where we need to deal with separate implicit imports for Oracle vs Postgres/H2
oracle implicit conversions for custom arrays and other problematic types
Do not override default debug level for all tests in participant-integration-api
CHANGELOG_BEGIN
Ledger API and Indexer Oracle Support
CHANGELOG_END
passing TransactionWriterSpec
passing JdbcLedgerDaoCompletionsSpec JdbcLedgerDaoDivulgenceSpec
passing JdbcLedgerDaoContractsSpec
All Oracle tests passing apart from one post-commit validation test
* Remove JdbcLedgerDaoValidatedOracleSpec as this is only relevant for classic postgres-backed sandbox
* rebase to master -- offsets are now varchar2 rather than blob
* remove use of DBMS_LOB operations
* remove all greater than/less than variants for DBMS_LOB
* revert postgres files that need not be touched
* code review feedback : avoid code duplication
* avoid indirection in type names for oracle arrays
* code review: HexString implicit conversions are not needed
* code review: Oracle case is not yet implemented for appendonlydao
* code review: Oracle case is not yet implemented for appendonlydao (cleanup import)
* code review: revert files that should not be touched
* address code review feedback: db specific imports for command completion become part of queries
* code review: perform db-specific reserved word escape to avoid case match
* code review: remove all dbms_lob comparison operations
* use simpler insert into with ignore dupes hint for oracle
* code review: avoid db specific match case in events range, use db specific limitClause
* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle
* code review: restore group by on Binary and Array fields for H2 and Postgres, disable for Oracle
* code review: restore group by on binary and array fields for non-oracle dbs, honour the calculation of limit size from QueryParty.ByArith
* code review: honour the calculation of limit size from QueryParty.ByArith
* code review: drop user after oracle test
* code review: remove drop user as it throws errors due to dangling sessions
* code review: revert incorrectly changed postgres schema files
* code review: clean up TODOs
* Remove // before hostname for consistency with other oracle connection strings
* code review: unambiguously scope table column referenced in select and where queries
* code review: correct duplicate table alias
