* Use `extra` in the port file runner, rather than `temporary`.
* ledger-api-test-tool-on-canton: Use the port check runner.
Much simpler than the port file runner for our purposes.
* Replace `runner` with `runner_with_port_file`.
Rather than expecting a particular set of command-line-arguments, we use
templating.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename the `runner_with_port_check` target to the default.
* rewrite trigger docs to follow gsg
Per #10419 point 4, I've rewritten the Triggers section to build upon
the Getting Started Guide instead of inventing its own example.
Compared to #10395, this has a lot more explanations as this page must
now serve the dual purpose of being a possible "next step" from the GSG
and being the main reference page for triggers. It's also lost the "next
steps" section, which I think is a bit of a shame, but it doesn't really
make sense here.
There's also no easy way for people not interested in the GSG to follow
along; should we expose the "completed GSG" as a tempate?
CHANGELOG_BEGIN
CHANGELOG_END
* keep copy-trigger as a template
* fix copy-trigger project name
* make up gsg-trigger template
* remove awkward sentence, fix existing typo
* update code to use when{,Some}
* add to
* swap emitCommands and getCommandsInFlight
* typo
* insist on state-correction perspective
* fix copy-trigger tests
* add back copy-trigger to whitelist
* add gsg-trigger to whitelist
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* Move ExceptionOps from ledger-service/utils to //libs-scala/scala-utils
* extract connection and JdbcConfig from //ledger-service to independent db-utils module
Changelog_begin
Changelog_end
* update trigger service to use new libs-scala/db-utils
* missed changes for http-json-oracle
* minor cleanup based on comments
* fix breaking scala 2_12 build
* cleanup db-utils/BAZEL.md file
* participant-integration-api: Encapsulate the initial configuration.
* participant-integration-api: Reduce usage of `LedgerConfiguration`.
* Inline `LedgerConfiguration` wherever it's used.
Most things don't need all its constituent parts; this reduces the
amount of unused properties.
CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
removed in favor of ``InitialLedgerConfiguration``. Its usage
has been changed accordingly, with the ``configurationLoadTimeout``
property becoming part of ``ApiServerConfig`` instead.
The default options provided by ``LedgerConfiguration`` have been
removed; you are now encouraged to come up with sensible values for
your own ledger. The ``Configuration.reasonableInitialConfiguration``
value may help.
CHANGELOG_END
* Correct the initial configuration submission delay for KV ledgers.
* kvutils: Mark supertype unused parameters as unused.
* kvutils: Extract out common configuration submission delays.
These values are specific to kvutils; other drivers should come up with
their own.
* configuration: Delete `NoGeneration`, as it's unused.
Was curious if there were any relevant performance improvements in
newer versions. Looks like the answer is no but we might as well
upgrade anyway.
changelog_begin
changelog_end
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.
CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
CHANGELOG_END
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
* logging-entries: Split from contextualized-logging.
This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Fix dependencies for 2.12.
* logging-entries: Missed one more Scala 2.12 dependency.
* release: Publish logging-entries.
* contextualized-logging: Automatically convert logging values to strings.
For now, this has almost the same behavior, but it allows us to
customize the output in the future.
The main change is that the log format has gone from:
context: {a=b, x=1, foo=bar}
to:
context: {a: "b", x: "1", foo: "bar"}
* contextualized-logging: Move `writeTo` inside `LoggingValue`.
* contextualized-logging: Allow for more than just strings.
`null`, numbers, and sequences are now correctly logged.
The log format has gone from:
context: {a: "b", x: "1", foo: "bar", parties: "[alice, bob]"}
to:
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_BEGIN
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_END
* contextualized-logging: Extract the string serializer.
* Use non-string logging where possible.
* contextualized-logging: Split logging values from serialization.
So that callers don't have to know about Jackson.
* contextualized-logging: `SeqView` is `Iterable`. Don't need both.
* contextualized-logging: Make `ToStringToLoggingValue` a `val`.
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* contextualized-logging: Add a transient dependency for 2.12 only.
This required more infrastructure than I thought it would.
* kvutils: Make it explicit that we're logging the hashes of archives.
The implicit was found to be a little confusing.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* upgrade scalacheck to 1.14.3
* regenerate maven_install files
* some different names and implicits
* remove some fromTryCatchNonFatal
* more porting
* port fromTryCatchNonFatal to attempt
* factor the assertions in SignatureSpec to avoid \/
* deal with invariant \/
* make partial unification do what we want
* \/, parse*, and toNel
* many uses of the .right method
* a legitimate use of fromTryCatchThrowable
* rebuild maven pins
* further invariant \/
* OneAnd and Nel interface changes
* further Either games
* \/ and reformatting
* \/ in http-json
* \/ in http-json
* deprecations
* more invariance
* cleanup unused
* more invariance; http-json compiles
* final either follies
* small 2.12 extra incompatibility
* rebuild deps
* revisit a couple earlier fixes using nicer expressions I learned later
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* repin 2.12
changelog_begin
- [Ledger HTTP Json Service] Logging output can now be in JSON either via providing the cli option `--log-encoder json` or via setting the env var `LOG_FORMAT_JSON=true`
changelog_end
* Make DA.Assert throw AssertionFailed instead of GeneralError
changelog_begin
- [Daml Standard Library] `assert`, `(===)`, and other assertion functions (see DA.Assert) now use a new `CanAssert` typeclass constraint instead of `CanAbort`, in preparation for exceptions support.
changelog_end
* Add CanAssert instances for Script and Trigger.
* buildifier-fix
* update script test runner output
* Fix flag and add tests.
* update script test runner again
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* participant-integration-api: Build Oracle tests, but don't run them.
CHANGELOG_BEGIN
CHANGELOG_END
* triggers: Switch to an environment variable for enabling Oracle tests.
* http-json: Switch to an environment variable for enabling Oracle tests.
* Disable running Oracle tests by default, not building them.
* triggers/service: Remove unused test dependencies.
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Set supported jdbc driver names at compile time
This is mainly to unblock the work on Oracle support in the Ledger API
but I think it’s a sensible thing in general. For the Ledger API,
moving the dependency to the top-level is apparently rather
tricky. Because the SDK bundles everything into a single megajar,
Sandbox depending on the oracle library does also result in the JSON
API and the trigger service will also have the oracle library in scope
and will support Oracle in CE which they should not.
This PR simply hardcodes the list of supported drivers to address
that. Not pretty but does the job.
changelog_begin
changelog_end
* format
* Address review comments
changelog_begin
changelog_end
* Expose tho Oracle support in the EE trigger service
This PR builds on the previous PR that did all the actual work on
Oracle support and exposes it in the enterprise edition. This PR only
releases the enterprise edition via the SDK tarball. I’ll add
artifactory publishing separately.
changelog_begin
changelog_end
* Update daml-assistant/daml-sdk/validate.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Add Oracle support in the trigger service
This PR migrates the ddl & queries and adds tests for this. It does
not yet expose this to users. I’ll handle that in a separate PR.
changelog_begin
changelog_end
* use getOrElse
changelog_begin
changelog_end
* Bump ghcide
Includes https://github.com/digital-asset/daml-ghcide/pull/13 meaning
we can now remove the hacks for missing signatures
changelog_begin
- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
produced errors for warnings produced by `-Wmissing-signatures` even
if the user did not explicitly enable this.
changelog_end
* Bump to merged commit
changelog_begin
changelog_end
We used to do this for some packages but it broke cpp. I don’t
actually know why it doesn’t do that anymore but I’ll gladly accept
that fact and turn it on everywhere.
changelog_begin
changelog_end
* Use DA.Map in triggers if available
Replaces DA.Next.Map by DA.Map in the trigger library if the DAML-LF
version supports it, i.e. above 1.11.
Selects on `DAML_GENMAP` and `DAML_GENERIC_COMPARISON` using `CPP`.
changelog_begin
- [Triggers] The trigger library now uses `DA.Map` instead of the
deprecated `DA.Next.Map` if the targeted Daml-LF version supports it.
This is a breaking change: Code that interfaced with the triggers
library using `DA.Next.Map`, e.g. with
`Daml.Trigger.getCommandsInFlight` or `Daml.Trigger.Assert.testRule`,
will need to be changed to use `DA.Map` instead.
changelog_end
* Deduplicate GMap imports in triggers
* DA.Next.Map --> DA.Map in carbon-upgrade-trigger
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
CHANGELOG_BEGIN
- [LF] Release LF 1.12. This version reduce the size of transaction
- [Compiler]: Change the default LF output from 1.8 to 1.11.
CHANGELOG_END
The jdkLogHandler provided by Doobie exists purely as an example and the library
itself does not recommend using it in production.
Note that this slightly changes the runtime behavior, logging successful queries
at debug level rather then info. The message itself is preserved from the original
MIT-licensed example.
This uses Slf4j as most of our components, instead of java.util.logging.
changelog_begin
[HTTP JSON API] The server now logs successful queries at debug level
instead of info
[Trigger Service] The server now logs successful queries at debug level
instead of info
changelog_end
* Use cliopts.Http for OAuth 2.0 middleware
changelog_begin
- [OAuth 2.0 Middleware] You can now configure the address that the
middleware listens to using the ``--address`` flag.
The port that the middleware listens to is now configured using the
``--http-port`` flag, use 0 to dynamically choose a free port.
You can now configure a port file where the chosen port will be
written to using the ``--port-file`` flag.
changelog_end
* Add test-case for OAuth 2.0 middleware port file
* Don't forget to close source
* Fix integration test
* Update triggers/service/auth/src/main/scala/com/daml/auth/middleware/oauth2/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* add livez endpoint to auth middleware
* Add OAuth 2.0 middleware to Daml SDK
* unhide trigger service auth flags
changelog_begin
- [Triggers] The trigger service now supports authorization through an
auth middleware. The feature is enabled using the `--auth` and
`--auth-callback` command-line flags. Please refer to the
Authorization chapter of the trigger service documentation for further
instructions.
- [OAuth 2.0 middleware] Daml Connect now includes an implementation of
the auth middleware API that supports OAuth 2.0 Authorization Code
Grant. Please refer to the Auth Middleware and OAuth 2.0 Auth
Middleware chapters of the documentation.
changelog_end
* drop early access flag on triggers
Daml triggers, the trigger service, and the auth middleware are no
longer marked as early access features.
changelog_begin
- [Triggers] Daml Triggers and the Trigger Service are no longer in
early access status.
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Include auth challenge in response body
changelog_begin
changelog_end
Some browsers make it difficult to access the `WWW-Authenticate`
response header from Javascript. For example, Firefox 84.0.2 (64-bit) on
Linux does not expose the `WWW-Authenticate` header in the result of the
`fetch` function, independent of the server's access control headers.
In that case the header is only accessible through the `XMLHttpRequest`
API, which is more cumbersome to use.
This adds the challenge to the response body in JSON format as well to
avoid exposing users to any such browser related issues.
* Make Secure Set-Cookie attribute configurable
changelog_begin
changelog_end
Since Chrome 80 Set-Cookie with the Secure attribute enabled is rejected
for connections that don't use https [1]. This includes localhost.
Firefox, at least as of version 84.0.2, allows such cookies on
localhost.
This adds a command-line flag to the authorization middleware to make
the value of the Secure attribute configurable. This way it can be
disabled for development purposes.
[1]: https://blog.chromium.org/2019/10/developers-get-ready-for-new.html
* Expand description on --cookie-secure flag
* Fix TriggerServiceFixture
* Fix middleware TestFixture
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Don't always redirect to /login automatically
The redirect mode can be configured to never redirect, always redirect,
or redirect based on the request type (redirect for text/html).
In case of no redirect the auth middleware client will reply with 401
Unauthorized with a custom WWW-Authenticate challenge to login on the
auth middleware.
* Make login to redirect configurable on trigger service
By default the trigger service will redirect for HTML requests and not
redirect for JSON requests. The test suite uses automatic redirect as
the OAuth2 test server works without user interaction.
changelog_begin
changelog_end
* Preserve path and query in authMiddlewareUri
This is necessary if the auth middleware lies behind a reverse proxy
with a path prefix or a similar setup.
* Bump default auth middleware login timeout
One minute was to short for a login cycle that requires manual user
input.
* Set token cookie properties
`path = "/"` is required so that the `/login` endpoint can reliably
override the cookie value for other endoints such as `/cb` or
`/v1/triggers`.
* Test redirectToLogin modes
* Redirect on HTML
https://github.com/digital-asset/daml/pull/8532#discussion_r559368335
* Use pass/reject in onRedirectToLogin
8db2bff9af (r559370308)
* default login timeout 5min
https://github.com/digital-asset/daml/pull/8532#discussion_r559535511
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Remove redundant list of LF versions
After #8472, I realized that there must be a list used for daml-stdlib
and daml-prim already and it turns out there is. I’ve removed that one
in favor of the one added in #8472 since I like having all in one
place and the one from #8472 is created by filtering an existing list
instead of creating a completely separate list like we do here.
changelog_begin
changelog_end
* Introduce SCRIPT_LF_VERSIONS
changelog_begin
changelog_end
* Use ActorSystem and ExecutionContext from RequestContext
* Factor out middleware server class
To avoid passing around config and state manually.
changelog_begin
changelog_end
* Depend on databricks/sjsonnet
changelog_begin
changelog_end
* Generate request params from jsonnet template
changelog_begin
changelog_end
* Split middleware test suite sources
* Add test suite for request templates
* fmt
* TriggerServiceFixture template arguments
* Use null to indicate missing applicationId claim
Addressing
https://github.com/digital-asset/daml/pull/8453/files#r555025173
* Fix invalid path on Windows
* Close request template source
* Avoid repeated re-reading of Jsonnet files
https://github.com/digital-asset/daml/pull/8453/files#r555044262
* Factor out template argument mappings
* factor out template error handling
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Bounded auth middleware client callback store
changelog_begin
changelog_end
* Json format for login response
* Add middleware client binding to test fixture
* Test middleware client callback store size
* Make max auth callbacks and timeout configurable on trigger service
changelog_begin
changelog_end
* Bounded pending login requests at auth middleware
* Make max logins and timeout configurable on middleware
* Test middleware login store size
* fmt
* Fix Windows
failed with `"localhost" != "127.0.0.1"`. Hardcode "localhost" to avoid
platform specific resolution.
* Use FiniteDuration for login timeout
* Respond with 503 on full login request store
Addresses https://github.com/digital-asset/daml/pull/8351#pullrequestreview-560859604
* Add tests for RequestStore
* Lower DefaultMaxLoginRequests
b48050eb91 (r552649275)
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>