* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
At the moment, the OAuth2 Middleware prints its config at startup.
Unfortunately that config includes a secret, which presumably we'd
prefer not to show.
CHANGELOG_BEGIN
- The OAuth2 Middleware now obfuscates its Client Secret when logging
its config.
CHANGELOG_END
Fixes#10957
changelog_begin
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
changelog_end
In many network setups, there will be a more direct route for the
trigger service to contact the auth middleware than going back through
the frontend's public IP address (and possibly thus also through
intermediaries like an nginx reverse proxy etc.). In _some_ network
setups, it may not even be possible for the trigger service to reach the
auth middleware through its externally-visible address.
This PR caters to these cases by allowing the trigger service to use two
separate addresses for the auth middleware, an internal one the trigger
service uses when it needs to talk to the auth middleware, and an
external one used in generating URLs for external clients.
This is backwards-compatible: if the old option is used, we simply use
the same value for both.
CHANGELOG_BEGIN
- The Trigger Service can now accept separate `--auth-internal` and
`--auth-external` CLI arguments, where `--auth-internal` is the
address used by the Trigger Service to reach the Auth Middleware
directly, and `--auth-external` is the address the Trigger Service uses
in generated URLs sent back to the client. The `--auth` option remains
and keeps working as before, setting both internal and external
addresses to the same given value.
CHANGELOG_END
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox
CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
Following #10763, we drop the ad-hoc builders for `FrontStack`.
* Building a `Fronstack` from individuals elements should be done with
standard scala buidler.
* Building a `Fronstack` from a `TraversableOne` should be done with
the scala 2.13 `.to(FrontStack)` methd
* Building a `Fronstack` from a `ImmArray` should be done with the
`toImmArray` method.
CHANGELOG_BEGIN
CHANGELOG_END
It seems a bit silly that the auth middleware would refuse a token that
has an `actAs` when it needs a `readAs`.
Reference: https://docs.daml.com/app-dev/authorization.html#access-tokens-and-claims
CHANGELOG_BEGIN
- The auth middleware will now consider that actAs credentials are valid
for readAs requests.
CHANGELOG_END
The workaround for https://github.com/akka/akka/issues/29841 is no
longer required as that issue has been fixed upstream and we have
updated to a version that includes the fix.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Test case for LockedFreePort not colliding with port 0
changelog_begin
changelog_end
* Discover dynamic port range on Linux
* Random port generator outside ephemeral range
* remove dev comments
* Draw FreePort from outside the ephemeral port range
Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.
LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.
But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.
* Select dynamic port range based on OS
* Detect dynamic port range on MacOS and Windows
* Import sysctl from Nix on MacOS
changelog_begin
changelog_end
* Windows line separator
* FreePort helpers visibility
* Use more informative exception types
* Use a more light weight unit test
* Add comments
* Fix Windows
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add a comment to clarify the generated port range
* fmt
* unused import
* Split libs-scala/ports
Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
The claims check in the auth middleware was switched around: in effect,
it checked that we did not receive _more_ than we asked for, rather than
checking we receive _at least_ what we asked for.
Of course this would still not let anyone run any trigger without the
proper access token, but it would let people list running triggers and
request (or stop) trigger executions.
CHANGELOG_BEGIN
- Fix a bug in the auth middleware where insufficient credentials could
still give access to list of running triggers.
CHANGELOG_END
* Do not require redirect URI upfront
changelog_begin
changelog_end
This allows users to configure the redirect URI after the middleware
client binding occurred such that dynamic port selection can be taken
into account.
* Use port 0 in trigger service fixture
changelog_begin
changelog_end
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update triggers/service/auth/src/test/scala/com/daml/auth/middleware/oauth2/TestMiddleware.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* unconditionally enable JSON search index on Oracle
In '1kb of data' and larger Oracle integration tests:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-20000: Oracle Text error:
DRG-50943: query token too long on line 1 on column 3
From https://docs.oracle.com/en/database/oracle/oracle-database/19/errmg/DRG-10000.html#GUID-46BC3B3F-4DB7-4EB4-85DA-55E9461966CB
Cause: A query token is longer than 256 bytes
Action: Rewrite query
* add changelog
CHANGELOG_BEGIN
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
CHANGELOG_END
* test only 1kb
* extra flag in db config string
* let Queries backends configure themselves from maps
* new Queries constructor dataflow to better support config values
* remove fields as we go, isolating backend-specific from -agnostic conf
- we use StateT to avoid the problems that will definitely arise if we
don't DRY.
* fix up DbConfig including DbStartupMode
* start to uncouple json-api's config from db-utils
* two JdbcConfigs with different purposes/scopes
- also moves db-utils contents to com.daml.dbutils
* adapt trigger service to refactoring
* fix JdbcConfig leftovers
* adapt http-json-cli to new JdbcConfig
* remove extra ConfigCompanion
* explain more about the QueryBackend/Queries distinction
* split SupportedJdbcDriver into two phases with a tparam
* use SupportedJdbcDriver.TC instead of SupportedJdbcDriver as the nullary typeclass
* patch around all the moved objects with imports
* missed import from moving ConnectionPool to dbutils
* use new 2-phase SupportedJdbcDriver for ContractDao setup
* left off part of a comment
* more q.queries imports
* other imports from the dbutils move
* nested JdbcConfig
* configure the driver in each backend-specific test
* very confusing error, but make the imports nicer and it goes away
* nested JdbcConfig in perf
* missing newline
* port contractdao-bench
* test new option parsing all the way through QueryBackend
* disable search index for some tests, enable for others
* add changelog
CHANGELOG_BEGIN
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
CHANGELOG_END
* port failure tests
* init version table last, drop first
- suggested by @realvictorprm; thanks
* rename split DBConfig.scala
- suggested by @realvictorprm; thanks
* move imports to not be in alphabetical order
- suggested by @realvictorprm; thanks
* remove createSchema
- suggested by @realvictorprm; thanks
* Revert "test only 1kb"
This reverts commit 616e173e63.
* port to scala 2.12
- bug in unused imports
- old name `-` for `removed`
Create normalized TXs when a partial TX is finalised.
Except in limited cases! (i.e for scenario-runner, sandbox)
CHANGELOG_BEGIN
CHANGELOG_END
normalize values in the engine as they are converted from speedy-values
fix 2.12 build
backout redundant change
ensure byKey field is correctly normalized when constructed by engine
rename flag: valueNormalization -> transactionNormalization
improve comment
delete commented-out code
rename: toValueNorm --> toNormalizedValue
rename: (SValue.) toValue --> toUnNormalizedValue
revert changes to ptx so that the interface to insertCreate() etc is Value-based (not SValue-based)
improve comments
respell: toUnNormalizedValue --> toUnnormalizedValue
fix build
This PR makes possible to check for contract IDs suffix during
preprocessing.
This is the first part of the task 3 described in #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* Use `extra` in the port file runner, rather than `temporary`.
* ledger-api-test-tool-on-canton: Use the port check runner.
Much simpler than the port file runner for our purposes.
* Replace `runner` with `runner_with_port_file`.
Rather than expecting a particular set of command-line-arguments, we use
templating.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename the `runner_with_port_check` target to the default.
* rewrite trigger docs to follow gsg
Per #10419 point 4, I've rewritten the Triggers section to build upon
the Getting Started Guide instead of inventing its own example.
Compared to #10395, this has a lot more explanations as this page must
now serve the dual purpose of being a possible "next step" from the GSG
and being the main reference page for triggers. It's also lost the "next
steps" section, which I think is a bit of a shame, but it doesn't really
make sense here.
There's also no easy way for people not interested in the GSG to follow
along; should we expose the "completed GSG" as a tempate?
CHANGELOG_BEGIN
CHANGELOG_END
* keep copy-trigger as a template
* fix copy-trigger project name
* make up gsg-trigger template
* remove awkward sentence, fix existing typo
* update code to use when{,Some}
* add to
* swap emitCommands and getCommandsInFlight
* typo
* insist on state-correction perspective
* fix copy-trigger tests
* add back copy-trigger to whitelist
* add gsg-trigger to whitelist
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils
* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end
* update trigger service to use new libs-scala/db-utils
* missed changes for http-json-oracle
* minor cleanup based on comments
* fix breaking scala 2_12 build
* cleanup db-utils/BAZEL.md file
* participant-integration-api: Encapsulate the initial configuration.
* participant-integration-api: Reduce usage of `LedgerConfiguration`.
* Inline `LedgerConfiguration` wherever it's used.
Most things don't need all its constituent parts; this reduces the
amount of unused properties.
CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
removed in favor of ``InitialLedgerConfiguration``. Its usage
has been changed accordingly, with the ``configurationLoadTimeout``
property becoming part of ``ApiServerConfig`` instead.
The default options provided by ``LedgerConfiguration`` have been
removed; you are now encouraged to come up with sensible values for
your own ledger. The ``Configuration.reasonableInitialConfiguration``
value may help.
CHANGELOG_END
* Correct the initial configuration submission delay for KV ledgers.
* kvutils: Mark supertype unused parameters as unused.
* kvutils: Extract out common configuration submission delays.
These values are specific to kvutils; other drivers should come up with
their own.
* configuration: Delete `NoGeneration`, as it's unused.
Was curious if there were any relevant performance improvements in
newer versions. Looks like the answer is no but we might as well
upgrade anyway.
changelog_begin
changelog_end
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.
CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
CHANGELOG_END
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
* logging-entries: Split from contextualized-logging.
This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Fix dependencies for 2.12.
* logging-entries: Missed one more Scala 2.12 dependency.
* release: Publish logging-entries.
* contextualized-logging: Automatically convert logging values to strings.
For now, this has almost the same behavior, but it allows us to
customize the output in the future.
The main change is that the log format has gone from:
context: {a=b, x=1, foo=bar}
to:
context: {a: "b", x: "1", foo: "bar"}
* contextualized-logging: Move `writeTo` inside `LoggingValue`.
* contextualized-logging: Allow for more than just strings.
`null`, numbers, and sequences are now correctly logged.
The log format has gone from:
context: {a: "b", x: "1", foo: "bar", parties: "[alice, bob]"}
to:
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_BEGIN
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_END
* contextualized-logging: Extract the string serializer.
* Use non-string logging where possible.
* contextualized-logging: Split logging values from serialization.
So that callers don't have to know about Jackson.
* contextualized-logging: `SeqView` is `Iterable`. Don't need both.
* contextualized-logging: Make `ToStringToLoggingValue` a `val`.
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* contextualized-logging: Add a transient dependency for 2.12 only.
This required more infrastructure than I thought it would.
* kvutils: Make it explicit that we're logging the hashes of archives.
The implicit was found to be a little confusing.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* upgrade scalacheck to 1.14.3
* regenerate maven_install files
* some different names and implicits
* remove some fromTryCatchNonFatal
* more porting
* port fromTryCatchNonFatal to attempt
* factor the assertions in SignatureSpec to avoid \/
* deal with invariant \/
* make partial unification do what we want
* \/, parse*, and toNel
* many uses of the .right method
* a legitimate use of fromTryCatchThrowable
* rebuild maven pins
* further invariant \/
* OneAnd and Nel interface changes
* further Either games
* \/ and reformatting
* \/ in http-json
* \/ in http-json
* deprecations
* more invariance
* cleanup unused
* more invariance; http-json compiles
* final either follies
* small 2.12 extra incompatibility
* rebuild deps
* revisit a couple earlier fixes using nicer expressions I learned later
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* repin 2.12
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
* Make DA.Assert throw AssertionFailed instead of GeneralError
changelog_begin
- [Daml Standard Library] `assert`, `(===)`, and other assertion functions (see DA.Assert) now use a new `CanAssert` typeclass constraint instead of `CanAbort`, in preparation for exceptions support.
changelog_end
* Add CanAssert instances for Script and Trigger.
* buildifier-fix
* update script test runner output
* Fix flag and add tests.
* update script test runner again
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* participant-integration-api: Build Oracle tests, but don't run them.
CHANGELOG_BEGIN
CHANGELOG_END
* triggers: Switch to an environment variable for enabling Oracle tests.
* http-json: Switch to an environment variable for enabling Oracle tests.
* Disable running Oracle tests by default, not building them.
* triggers/service: Remove unused test dependencies.
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Set supported jdbc driver names at compile time
This is mainly to unblock the work on Oracle support in the Ledger API
but I think it’s a sensible thing in general. For the Ledger API,
moving the dependency to the top-level is apparently rather
tricky. Because the SDK bundles everything into a single megajar,
Sandbox depending on the oracle library does also result in the JSON
API and the trigger service will also have the oracle library in scope
and will support Oracle in CE which they should not.
This PR simply hardcodes the list of supported drivers to address
that. Not pretty but does the job.
changelog_begin
changelog_end
* format
* Address review comments
changelog_begin
changelog_end
* Expose tho Oracle support in the EE trigger service
This PR builds on the previous PR that did all the actual work on
Oracle support and exposes it in the enterprise edition. This PR only
releases the enterprise edition via the SDK tarball. I’ll add
artifactory publishing separately.
changelog_begin
changelog_end
* Update daml-assistant/daml-sdk/validate.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add Oracle support in the trigger service
This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.
changelog_begin
changelog_end
* use getOrElse
changelog_begin
changelog_end
* Bump ghcide
Includes https://github.com/digital-asset/daml-ghcide/pull/13 meaning
we can now remove the hacks for missing signatures
changelog_begin
- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
produced errors for warnings produced by `-Wmissing-signatures` even
if the user did not explicitly enable this.
changelog_end
* Bump to merged commit
changelog_begin
changelog_end
