* WIP
* First version that compiles (with debug print statements)
* Revert changes on the failure tests
* Make all tests pass with some major cleanup of the new code
* Remove debug prints & outcommented code
* Make decoding / encoding test also compatible with user token tests
changelog_begin
changelog_end
* Minimize diff
* Fix oracle tests
* leave note for followup
* Update ledger-service/http-json/src/itlib/scala/http/AbstractHttpServiceIntegrationTest.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Fix duplicate calls to the command service while debug logging is enabled
changelog_begin
- [HTTP-JSON] Fixed duplicate calls to the command service while debug logging is enabled.
- [HTTP-JSON] Fixed internal server errors which can occur while debug logging is enabled.
changelog_end
* Cleanup the code a bit, test still failing tho :(
* Finally fix the bug for real now
* Address review comments
* don't cast
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* move user management endpoint functions to separate module
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* move create and exercise endpoints to separate module
* fix copyright headers
* mark the helpers package-private
* more privacy modifiers
* import cleanup
* move Forwarded header test to unit tests
* move decodeAndParseUserIdFromToken to UserManagement
* move package endpoints to separate module
Enriches #12766 with a chunked, lazy internal approach to
aggregating pages from the participant server, rather than
aggregating all users directly in memory before returning them.
CHANGELOG_BEGIN
- [JSON API] If ``/v1/users`` fails while retrieving users from the participant,
it will be reported as a warning, with a partial set of users, rather than
failing.
See `issue #12777 <https://github.com/digital-asset/daml/pull/12777>`__.
CHANGELOG_END
* SandboxNextFixture replaced by Sandbox-on-X based SandboxFixture
changelog_begin
changelog_end
* Some fixed tests
* No direct dependencies on //ledger/sandbox:sandbox and //ledger/sandbox:sandbox-scala-tests-lib
* Fix after rebase
* Rename SandboxFixture and add a missing dep
* Generate valid party names if hint is empty
* Smaller maxInboundMessageSize
* Added test for empty display name
* SandboxServer is a ResourceOwner
* Uses execution context passed as an input for resource management
* Fixes flaky FlywayMigrations issue with null Thread.currentThread.currentClassLoader
* SandboxServer simplification returns Port instead of ApiServer
* Dedicated PMAllocateWithoutDisplayName for non-Canton ledgers
* Created since Canton does not return empty display names
* Return all users from `GET /v1/users/`, paginating 1k at a time
The test runs overall for ~30 seconds, mostly due to the time required to
create 20k users.
changelog_begin
[HTTP-JSON API] Previously, a 10k limit was in place on the number of users returned by
`GET /v1/users/`. This limit has been removed and users are retrieved in chunks from the ledger
changelog_end
As of now, this builds the response in-memory, which is likely not what we want to
support long-term.
* Relax test condition due to interference with other test setups
* Use fetchAndPersistBracket instead of fetchAndPersist
* add changelog
CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a POST-style multi-template query backed by
database could have different template IDs appear to be queried at different
ledger times, if updated concurrently. This conditions is now checked and
accounted for. See `issue #12672
<https://github.com/digital-asset/daml/pull/12672>`__.
CHANGELOG_END
* add a searchQuery metric event for each fetch trial in sync search
* revert the contract ID and contract key fetches, and explain why
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* upgrade scalapb/netty/grpc/protobuf in proven combination
CHANGELOG_BEGIN
Upgrade scalapb, netty, grpc, protobuf and guava versions
CHANGELOG_END
* bazel reformat
* match grpc version in deps.bzl
* upgrade akka
* keep grpc version to 1.43 that is used in latest nixpkgs-unstable
* rebase and regen
* update SHA for scalapb tarball
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
For consistency with other APIs in this area.
Note that some pre-existing APIs use `List` instead of `Seq`,
but at least those use the same underlying implementation.
CHANGELOG_BEGIN
CHANGELOG_END
* Return an empty object for deleteUser & createUser
* Make the field rights for creating a user optional & fix the tests
changelog_begin
- [HTTP-JSON] Adjusted the response for the createUser & deleteUser endpoint to an empty object instead of "true". This was done to allow possible future extensions to the return type without applying breaking changes.
- [HTTP-JSON] Made the field rights optional for the request body of the createUser endpoint. Now it is possible to create a user with just the userId (i.e. { "userId": "nice.user" }).
changelog_end
* Fixes OracleAround so it creates unique oracle users
* Fixes rouge connection pool in JdbcLedgerDaoTransactionsSpec
* Fixes cleanup in OracleAroundAll
* Introduces lockIdSeed for test frameworks
* Adapts usage
changelog_begin
changelog_end
* Add an endpoint used for querying user rights of a specific user
changelog_begin
- [HTTP-JSON] Added endpoint /user/rights that if called with POST will return user rights of the user specified via the payload
changelog_end
* Add grant&revoke user rights endpoint
changelog_begin
- [HTTP-JSON] Added endpoints user/rights/grant & user/rights/revoke which allow granting & revoking user rights for a specific user
changelog_end
* Switch to using the UserRights list instead of a case class with fields
* fix type annotation
* Rename allocateUser func to createUser to comform with the endpoint path
* Response format simplification & cleanup
* Add one last test that ensure that the JSON format of UserRight doesn't change without us noticing
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Remove unnecessary conversations and also just use unwrap for tagged strings
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.
[Here][1] is s a blog post by the Scala team about it.
Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.
Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.
changelog_begin
changelog_end
[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
changelog_begin
- [HTTP-JSON] Removed the /user/delete GET endpoint. Please use the /user/delete POST endpoint with the own user ID if you need to delete the user associated with the current token
changelog_end
* Split channel configuration from LedgerClientConfiguration
Fixes#12391
The channel configuration now has to be provided separately from the
configuration specific to the ledger client. In this way we avoid
situations where the builder is provided with some configuration
that gets overridden.
changelog_begin
[Scala bindings] The channel configuration has been split from the
LedgerClientConfiguration class. Provide the gRPC channel specific
configuration separately or use a builder. The channel configuration
no longer overrides the builder.
changelog_end
* Fix compilation issues in //ledger-service/...
changelog_begin
- [HTTP-JSON] Added endpoints:
- /user/delete that if called with GET will delete the current user & with POST will delete the user specified via the payload
- /user that if called with POST will now return user info about the user specified via the payload
changelog_end
* Add list users endpoint
changelog_begin
- [HTTP-JSON] Added an endpoint /users which returns the available users on the ledger.
changelog_end
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/domain.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Change heartBeatPer to more intuitive naming of heartbeatPeriod
CHANGELOG_BEGIN
CHANGELOG_END
* Initial changes to add HOCON config for json_api
CHANGELOG_BEGIN
CHANGELOG_END
* avoid IllegalArgumentException noise
* use named arguments in big config conversion
* Changes include
- tests for a full http-json-api config file
- logging config and non-repudiation config is still specified via cli args.
- config readers for MetricsReporter
* Add defaults to WebsocketConfig case class to allow partially specifying fields on typeconf file
* changes to the JwtVerifierBase config reader and equivalent test
* message already describes the value
* replace manual succeed/fails with scalatest combinators
* use qualified imports for WebsocketConfig defaults
* add back autodeleted empty lines
* collapse two lists of token verifiers into one
* add new line to config files
* rename dbStartupMode to startMode to keep consistent with cli option and for easy documentation
* Changes to daml docs to specify ways to run JSON-API by supplying a HOCON config file.
CHANGELOG_BEGIN
JSON-API can now be started supplying a HOCON application config file using the `--config` option.
All CLI flags except `logging` and `non-repudiation` one's are now deprecated and will be cleaned up in some future releases.
CHANGELOG_END
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* WIP
* Adjust the format of the CreateUserRequest to be a nicer payload & add a simple test
changelog_begin
- [HTTP-JSON] An endpoint user/create has been added to be able to create a user via the json api
changelog_end
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Changelog entry and commit msg differ here because the bug described in
the changelog was already fixed by adding the user management support
because it caused for the affected endpoints that it will be interpret as
user token while only fetching the ledger id (without actually checking
that it is a user token).
changelog_begin
- [HTTP-JSON] Fixed a bug that caused jwt's without the daml namespace to be rejected for some endpoints (https://github.com/digital-asset/daml/issues/12215)
changelog_end
* Changes to add the option of starting trigger service with typeconf/HOCON config
CHANGELOG_BEGIN
CHANGELOG_END
* add tests for authorization config and fail on both config file and cli args
* refactor and cleanup config loading and tests
* Changes based on code review comments
* Daml doc changes and making sure that we have defaults for most fields to mirror cli args
CHANGELOG_BEGIN
Trigger Service can now be configured with HOCON config file.
- If a config file is provided we will choose to start the service using that, else we will fallback to cli arguments.
- If both config file and cli args are provided we will error out.
CHANGELOG_END
* addressing some more code review comments
* use scalatest inside properly
* Expose user management service over the HTTP-JSON API
Fixes#12078
Add user & user/rights endpoint which provide the current user id, primary party and the latter the user rights
Fix new endpoints for ledgers without auth and add test coverage for these
changelog_begin
- [HTTP-JSON] Added GET endpoint:
- /user which returns the current user id & primary party
- /user/rights which returns the user rights of the current user
changelog_end
* Update ledger-service/http-json/src/it/scala/http/HttpServiceIntegrationTestUserManagement.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Apply review comments
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/domain.scala
Co-authored-by: akshayshirahatti-da <86774832+akshayshirahatti-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: akshayshirahatti-da <86774832+akshayshirahatti-da@users.noreply.github.com>
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* WIP
* Remove the dummy implementation and replace it with an actual working implementation
* Make it compile!
* Add working tests for the user management support in the json api
CHANGELOG_BEGIN
- [JSON-API] Added basic support for the new user management feature of the ledger such that user tokens are now accepted instead of the legacy tokens
CHANGELOG_END
* Simplify the create iou test case and adjust the test case name to be correct
* Add additional test that covers that the overwrite of actAs&readAs still works via the meta object
* Make it work with unauthenticated ledgers too
* Fix compile error & wrong behaviour & add test coverage for non auth ledgers
* Clean up the diff
* Address 66312e9940 (r770782884)
* Address 66312e9940 (r770750653)
* Addressing 66312e9940 (r770751958)
* Address 66312e9940 (r770736671)
* Address 66312e9940 (r770734395) and 66312e9940 (r770783237)
Co-authored-by: Stefano Baghino <stefano.baghino@digitalasset.com>
CHANGELOG_BEGIN
- [User Management]: add support for managing participant node users and authenticating
requests as these users using standard JWT tokens.
CHANGELOG_END
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
Co-authored-by: Adriaan Moors <90182053+adriaanm-da@users.noreply.github.com>
* replace pour with a new, total, uncurried apply to create NonEmpty's
* use the new NonEmpty apply in place of pour
* non-empty cons, snoc, head, tail
* add map and flatMap for NonEmpty iterables
* remove scala-collection-compat from scalautils
* tests for map, flatMap, cons
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* missing 'extends AnyVal'
* colliding map and flatMap for Maps
* Revert "colliding map and flatMap for Maps"
* more specific Map and Set return types
* type tests for map operations
* add 'to' conversions
* Add query store metrics
CHANGELOG_BEGIN
- [JSON-API] added metrics to separately track:
- time taken to update query-store ACS (from ledger)
- lookup times for the query store
CHANGELOG_END
* Apply review comment
* [Self-service error codes] Enabled by default
* Flag changed to `use-pre-1.18-error-codes` (disabled by default)
CHANGELOG_BEGIN
[Ledger API Specification] The Ledger API returns enriched error codes (see https://docs.daml.com/error-codes/self-service/index.html)
For backwards-compatibility, a new API flag `--use-pre-1.18-error-codes` is introduced for preserving the legacy behavior for
clients that want to migrate incrementally to the changed gRPC status code responses and error details format.
CHANGELOG_END
* Adapted HttpServiceIntegrationTest
* Renamed `Feature Flag` to `Configuration` in docs
* Fix Daml Script tests
changelog_begin
changelog_end
* Fix Repl functests
changelog_begin
changelog_end
* Fix haskell binding tests
changelog_begin
changelog_end
* Fix CommandClientIT test
* Fixed Sandbox and CommandServiceBackpressureIT tests
Please enter the commit message for your changes. Lines starting
* Adapt //compiler/damlc/tests:repl-functests again
* Fix more tests and address Miklos' comments
* Flag name changed to `grpc-status-codes-compatibility-mode`
* Remove useless flags sandbox-classic
* Sandbox-classic tests fix for ContractKeysIT and ExceptionsIT
* Created 2 deprecated test suites that have the more generic assertions as returned
by the deprecated in-memory backend
* More fixes for CommandServiceIT
* Fixes compilation issue with the deprecated exceptionsIT class for Sandbox-classic in-memory
* Compatibility mode for old test tools
* Change flag name to `use-pre-1.18-error-codes`
* Apply suggestions from code review
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Update to Java 11
changelog_begin
changelog_end
* Fix RoundingMode deprecation warnings
* Fix dep-ann warning
* Integer constructor
* JavaX annotation dependency
* javax.xml.bind was removed in Java 11
Using Guava as a replacement, since it is already a project dependency.
* JDK 11 no longer has a separate JRE tree
* Remove unused jdk_nix import
* remove now redundant jdk11_nix
* Java 8 --> 9 increased Instant.now() precision
See https://bugs.openjdk.java.net/browse/JDK-8068730
The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.
`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.
* TMP round timestamps
* Revert "TMP round timestamps"
This reverts commit af8e261278.
* Skip versions before 1.6.0 in migration tests
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Changes to port JMH benchmark tests for contract dao to be be run against postgres db.
CHANGELOG_BEGIN
CHANGELOG_END
* add missing libs-scala/ports depenency to ledger-on-sql test suite
* changes based on codereview comments
* make http-json:integration-tests into test suites
* make http-json-oracle:integration-tests into test suite
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* remove commented libraries
* Changes to renable ws multiplexing
CHANGELOG_BEGIN
[TS-BINDINGS] Re-enable ws multiplexing for stream queries after resolving the reconnect connection close bug associated with ws state and liveness.
CHANGELOG_END
* websocket is passed as an argument to the onMessage handler
* consistently use 'manager' reference instead of 'this' in the handleQueries change method
* add actAs, readAs to `meta` for create, exercise, createAndExercise endpoints
* use meta actAs, readAs to control how contract IDs are looked up for exercise
* outdated comments on JwtWritePayload and JwtPayload
* toSet1 operator to clean up some NEL manipulation
* take optional readAs argument for query endpoint
* use readAs for query POST
* check whether readAs is safe in query endpoint
* missed CommandMeta args in tests
* FetchRequest, a domain model to layer on "fetch" endpoint's ContractLocator
- ContractLocator was overloaded as a domain request model *and* a component
of other domain request models; the addition of new arguments means it can
no longer exactly meet the former, and adding "readAs" to it would poison it
for the latter cases.
* take readAs argument from fetch endpoint
* add readAs security check from query to fetch
* move jwt parties functions to util
* testing the party-set/JWT functions
* missing headers
* caught boolean blindness in readAs security checks
* test that meta params are used for commands
* make resolveRefParties do a subset check, too
* Revert "make resolveRefParties do a subset check, too"
This reverts commit 40a66f102c.
* test that the readAs auth check actually applies
* test that command service uses meta readAs, actAs
* note on test coverage
* add changelog
CHANGELOG_BEGIN
- [JSON API] ``actAs`` and ``readAs`` may be specified for create, exercise,
create-and-exercise, non-WS fetch, and non-WS query.
See `issue #11454 <https://github.com/digital-asset/daml/pull/11454>`__.
CHANGELOG_END
* no saving mallocs
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* untabify
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* move design comment to comment from function name
- suggested by @cocreature; thanks
* remove unneeded variable
* refactor single-key callers of requestJsonReader
- suggested by @cocreature; thanks
* build error in ce
* diagnose Windows failure
* add missed http-json-testing requirement
* use readers as fetch/query party-set name
- suggested by @cocreature and @realvictorprm, thanks
* extra import
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Changes to make certain hikari cp connection pool properties configurable via jdbc conf string
CHANGELOG_BEGIN
[JSON-API] Make certain Hikari cp connection pool properties configurable via jdbc conf string, the properties are listed below
poolSize -- specifies the max pool size for the database connection pool
minIdle -- specifies the min idle connections for database connection pool
connectionTimeout -- long value, specifies the connection timeout for database connection pool
idleTimeout -- long value, specifies the idle timeout for the database connection pool
CHANGELOG_END
* some missed changes for DbTriggerDao
* remove defaults for poolSize on JdbcConfig
* add constants for test defaults
* move contract insertion/deletion batching to separate function
* limit contract insertion/deletion batching on backpressure
* add changelog
CHANGELOG_BEGIN
- [JSON API] While updating the contract table for a query, if the DB appears to be slow,
JSON API will slow down its own inserts and deletes at some point rather than construct
ever-larger INSERT and DELETE batch commands.
See `issue #11589 <https://github.com/digital-asset/daml/pull/11589>`__.
CHANGELOG_END
* add PartySet alias for db-backend
* add PartySet alias for fetch-contracts
* add PartySet alias for http-json
* deprecate old apply
* quick builder for NonEmpty collections
* replace PartySet in db-backend
* replace PartySet in fetch-contracts
* lar.Party is also domain.Party
* add incl1 operator
* replace PartySet in http-json
* port tests
* into with Scala 2.12 needs collection-compat
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* simplify a couple functions that don't need so much data transformation now
* clean up some OneAnds and HKTs
* deal with Scala 2.12 without having warning suppression
* better, more obscure choice for Scala 2.12
