* [DPP-417][DPP-613] Adopt self-service error codes ApiTransactionService
CHANGELOG_BEGIN
CHANGELOG_END
* TV review
* TV review
* Uses ErrorFactories
* Error loggers use enriched logging contexts
* Deleted ApiTransactionServiceSpec
* Add TODO wrt to CommandValidation.invalidArgument
* Adapt transactionNotFound error assertions in conformance tests
* ErrorFactories.invalidArgumentWasNotFound for handling invalid event ids
* Revert improvement to TransactionNotFound cause message since
Canton still outputs the old message and assertions fail.
Co-authored-by: Tudor Voicu <tudor.voicu@digitalasset.com>
* [Self-service error codes] Implement V2 in Authorizer
CHANGELOG_BEGIN
CHANGELOG_END
* Added unit test for authorize (non-streamed)
* Fix after rebase
* Do not expose the error codes switching mechanism to the Java bindings
* Adjust InternalAuthorizationError to be SystemInternalAssumptionViolated
* Parameter names in test
* Testing AuthorizationInterceptor with regard to returned error codes
* Do not use default error code version switchers at instance creation
* Addressed Pawel's review comments
* Using ErrorFactories for error dispatching
* Pass loggingContext to Authorizer where available
* Generic internal authorization error
* Moved ErrorCodesVersionSwitcher to //ledger/error
CHANGELOG_BEGIN
CHANGELOG_END
* Rename ErrorCodeLoggingContext to ContextualizedErrorLogger
* Refactored ErrorFactories
* All error factories use ContextualizedErrorLogger for being able to dispatch self-service error codes.
* The ContextualizedErrorLogger is passed down from the dispatching Ledger API services.
* ErrorFactoriesSpec asserts both legacy (V1) and self-service error codes (V2).
* Adapted ApiSubmissionService
* Addressed Marcin's review comments
* Rename Completion.deduplication_time to deduplication_duration
CHANGELOG_BEGIN
CHANGELOG_END
* Breaking protobuf change: regenerate `buf` image
Breaking-Proto: true
* Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result
The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.
CHANGELOG_BEGIN
CHANGELOG_END
* Do not call duplicate command exception twice
* Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness
CHANGELOG_BEGIN
CHANGELOG_END
* Make schema migration retries and backoff configurable
* Review feedback - use RetryStrategy.constant instead
* Remove unused tailrec import
* Simplifications and runF by Marton
* Rename config options from retry to attempt and default to 30 attempts
* Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing
* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Fix broken test and use consistency for tests
* ledger-api-test-tool - Add conformance test for parallel command deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Add import for 2.12 compat
* Add silencer plugin
* Split parallel command deduplication scenario into it's own test suite
* Add the parallel command deduplication test to append only ledgers
* Run parallel command deduplication tests for append only ledgers
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Code review renames
* Add compat import
* Run the test concurrently
* Further rename completions' `deduplication_time` to `*_duration` in participant's codebase
CHANGELOG_BEGIN
CHANGELOG_END
* Re-compute sha256s of migrations
* Fix expected error message
* Improve consistency of "deduplication time" Vs. "deduplication duration" references
* Revert Oracle schema whitespace change
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
Co-authored-by: nicu-da <nicu.reut@digitalasset.com>
Co-authored-by: Oliver Seeliger <46452754+oliverse-da@users.noreply.github.com>
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
99% of our usecases use Value[ContractId] so this PR just fixes it.
The few other usescases are:
1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.
We don’t have any code which benefits from being polymorphic in the
contract id type.
changelog_begin
changelog_end
* Create a new grpc exception for each duplicate result
The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.
CHANGELOG_BEGIN
CHANGELOG_END
* Do not call duplicate command exception twice
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox
CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
Following #10763, we drop the ad-hoc builders for `FrontStack`.
* Building a `Fronstack` from individuals elements should be done with
standard scala buidler.
* Building a `Fronstack` from a `TraversableOne` should be done with
the scala 2.13 `.to(FrontStack)` methd
* Building a `Fronstack` from a `ImmArray` should be done with the
`toImmArray` method.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
- [Ledger API Specification] `Commands.deduplication_time` field has been deprecated, please use `Commands.deduplication_duration` instead.
CHANGELOG_END
Since we switch to scala 2.13, ImmArray companion object extends
`Factory`. Hence:
- the `apply` methods of `ImmArray` override the one from `Factory`
- we can use the notation `.to(ImmArray)` to convert an `Iterable` to
`ImmArray`
This PR drops those `apply` ImmArray. Conversion from Iterable to
`ImmArray` should use the `.to(ImmArray)`.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-common: Replace `URL` with a trait so we can fake it.
This means we don't have to use Mockito to stub out `URL`, which is
final and therefore requires magic to stub.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-common: Move tests that open streams into SecretsUrlTest.
* ledger-api-common: Make SecretsUrl constructors simple methods.
* ledger-api-common: Reduce concrete implementations of SecretsUrl.
Adding support for accepting server's private key as an encrypted file (since storing unencrypted private key in a file system might be a risk).
Encrypted private key is assumed to be encrypted using AES or similar algorithm. The details necessary to decrypt it are be obtained from a secrets server over HTTP as JSON document. The URL to secret's server is supplied through the new `--secrets-url` CLI parameter.
One can supply private in either plaintext (old behavior) or ciphertext: if a private key's file ends with .enc suffix it is assumed to be ciphertext. Otherwise it is assumed to be plain text.
CHANGELOG_BEGIN
- [DPP-418] [Participant] Add support for supplying server's private key as an encrypted file and then decrypting it with the help of a secrets server.
CHANGELOG_END
* bindings-akka: Parameterize RetryInfo with its value type.
* ledger-api-client: Wrap command submissions in a new class.
This will allow me to pass a deadline alongside the commands in the
future.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-client: Inline `CommandRetryFlow.Value`.
* bindings-akka: Inline `CommandRetryFlow`'s `createRetry`.
It's always the same: `retryInfo.value`.
* Add `deduplication_duration` to `deduplication_period` and deprecate `deduplication_time`
CHANGELOG_BEGIN
ledger-api - add `deduplication_duration` as a future replacement for `deduplication_time` in the command proto definition
CHANGELOG_END
* Add tests for deduplication period validation
CHANGELOG_BEGIN
ledger-api - Command deduplication period can now be specified by setting `deduplication_offset` instead of `deduplication_time` (only valid for v2 WriteService). This change is backwards compatible.
CHANGELOG_END
* Propagate the enriched deduplicationPeriod instead of deduplication duration
* Update the Haskell bindings for the new deduplication period
* Calculate the deduplicateUntil using the new deduplication period for backward compat
* Use consistent naming for deduplication_period
* Cleanup command timeout extraction from deduplication period
* Add the required deduplication_offset to deduplication instead of deduplication_start
* Update haskell bindings to support deduplication_offset
* Add support for deduplication_offset in the ledger-api
* Remove the timestamp-based deduplication from our models to simplify upgrade for users
* Add optional conformance test for offset based deduplication
* Remove buf rule for FIELD_SAME_ONEOF as our change is backwards compatible
* Disable FIELD_SAME_ONEOF buf check for commands file
* Apply suggestions from code review
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Update comment for deduplication period
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* participant-integration-api: Make `TrackerMap` a `Tracker`.
Mostly by moving parameters around.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Clean up `TrackerMap` a little more.
* participant-integration-api: Make `TrackerMap` generic.
* participant-integration-api: Move `Tracker.WithLastSubmission`.
It's only used by `TrackerMap`, so now it's an internal class there.
* participant-integration-api: Only provide the key to `newTracker`.
* participant-integration-api: Subsume the `TrackerMap` cleanup schedule.
* participant-integration-api: Construct the tracker map outside.
* ledger-api-common: Remove some unnecessary braces.
* participant-integration-api: Prettify `TrackerMap` some more.
* participant-integration-api: Make `TrackerMap.selfCleaning` a class.
* participant-integration-api: Add some tests for TrackerMap.
* participant-integration-api: Convert a method to `Runnable`, 2.12-style.
Apparently underscores aren't good enough.
* ledger-api-client: Delete CompletionSuccess#unapply.
It doesn't work on Scala 2.12.
* Refactor ParameterStorageBackend
- a single method for atomic ledger initialization
- a single method to look up the ledger end
changelog_begin
changelog_end
* Add a test
* Fix reading event sequential ids
* Remove debug statements
* Allow ledgerEnd on an empty database
* Initialization is not safe to call concurrently
* Remove leftovers from isolation level change
* Use unit return type
for initialization methods
* Allow getParticipantId on an empty database
* Use exceptions instead of a return type ADT
* Don't use Try for initialization
* Clean up parameters table
* Simplify parameter storage api
* Address review suggestion
* Address review comment
* Address review comment
* Prefer ledger id over participant id mismatch
* Address review comment
* Move type definition
* Remove useleess new keyword
* Renove unused import
* Inline result mapping
* Fix reporting of mismatching participantId
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* Add optional submission id to commands.proto
This allows to propagate a submission id. If no id is submitted (the submission id is empty) then we generate a new submission id
CHANGELOG_BEGIN
Add optional submission_id to the commands.proto.
CHANGELOG_END
* Update haskell bindings to include the submission id
* Code review - rename submission id extractor
* Code review - update comment and remove braces from if block
* Fix braces
* participant-integration-api: Move test constants to a companion object.
TelemetrySpecBase doesn't need to expose those through inheritance.
* Avoid mocks in the configuration management service test.
* Add tests for `ConfigManagementService.GetTimeModel`.
* Add tests for `ConfigManagementService.SetTimeModel`.
* Disable configuration management until the ledger has a configuration.
CHANGELOG_BEGIN
- [API Server] The configuration management service previously returned
the participant-specified default configuration if none was found on
the ledger. This can be misleading, and so the ``GetTimeModel``
endpoint now returns a gRPC ``NOT_FOUND`` error if no configuration
has been found yet. Similarly, the ``SetTimeModel`` endpoint returns a
gRPC ``UNAVAILABLE`` error.
This should only happen if the participant gives up waiting for the
ledger to provide a configuration, which is very unlikely in a
production setting. In this case, the ledger will also not respond to
command submissions.
CHANGELOG_END
* participant-integration-api: Extract some test variables for clarity.
In ApiConfigManagementServiceSpec.
* sandbox: Improve some variable and parameter names in tests for clarity.
* Improve the error message in case of a missing ledger configuration.
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* ledger-api-common: Improve an error status constructor name.
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* participant-state: Give `ChangeId` its own file.
* participant-state: Introduce `v1.CompletionInfo` for a while.
`CompletionInfo` is identical to `SubmitterInfo`. Its purpose is to make
the migration to v2 easier. It should not make it into the final
version.
* participant-integration-api: Switch to the v2 participant state API.
This means that:
- the API server and indexer expect v2 API traits
- adapters are provided if you need to elevate your v1 API usage
- the indexer internally uses v2 Updates
- rejections are switched over to use the v2 format
- Sandbox Classic uses v2 as the internal representation too (because
it interacts directly with the underlying indexer representation,
and is therefore tightly coupled)
_kvutils_ and other users of the `StandaloneApiServer` and
`StandaloneIndexerServer` use the adapters.
CHANGELOG_BEGIN
- [Integration Kit] The API server and indexer have switched over to v2
of the participant-state API. You can continue to use the v1 API, but
you will need to wrap your ``ReadService`` and ``WriteService``
objects in the ``AdaptedV1ReadService`` and ``AdaptedV1WriteSerivce``
classes.
CHANGELOG_END
* participant-state: Remove v1.CompletionInfo.
It's served its purpose.
* kvutils: Remove an unnecessary line from `Runner`.
* ledger-api-common: Delete a TODO; we'll track it elsewhere.
* participant-integration-api: Use full words in `JdbcLedgerDao`.
Just a little bit of cleanup.
* ledger-api-common: Extract out the random submission ID generator.
And introduce a trait, because, well, this is still the JVM.
* ledger-api-domain: Store the deduplication duration in `Commands`.
We can compute the max deduplication time from this later.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Refactor out `Commands#deduplicateUntil`.
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.
This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Remove an unused import.
* http-json-oracle: Remove `ledger-api-common` as a dependency.
* bindings-rxjava: Remove a now-unused dependency.
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
* logging-entries: Split from contextualized-logging.
This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Fix dependencies for 2.12.
* logging-entries: Missed one more Scala 2.12 dependency.
* release: Publish logging-entries.
* contextualized-logging: Automatically convert logging values to strings.
For now, this has almost the same behavior, but it allows us to
customize the output in the future.
The main change is that the log format has gone from:
context: {a=b, x=1, foo=bar}
to:
context: {a: "b", x: "1", foo: "bar"}
* contextualized-logging: Move `writeTo` inside `LoggingValue`.
* contextualized-logging: Allow for more than just strings.
`null`, numbers, and sequences are now correctly logged.
The log format has gone from:
context: {a: "b", x: "1", foo: "bar", parties: "[alice, bob]"}
to:
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_BEGIN
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_END
* contextualized-logging: Extract the string serializer.
* Use non-string logging where possible.
* contextualized-logging: Split logging values from serialization.
So that callers don't have to know about Jackson.
* contextualized-logging: `SeqView` is `Iterable`. Don't need both.
* contextualized-logging: Make `ToStringToLoggingValue` a `val`.
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* contextualized-logging: Add a transient dependency for 2.12 only.
This required more infrastructure than I thought it would.
* kvutils: Make it explicit that we're logging the hashes of archives.
The implicit was found to be a little confusing.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* CHANGELOG_BEGIN
Log ledger api validation failures at info level
CHANGELOG_END
* log validation failures in Api*Service family of classes
* address review comments
Before this change, any write-service was restricted to the predefined
set of rejection reasons which couldn't capture the complex canton
rejection reasons.
In order to open up the path for a redesign, I've migrated the current
rejection reasons into a V0 version and opened the abstract class for
customized implementations. I've also added a temporary method to tunnel
synchronous rejections through the ledger api server.
CHANGELOG_BEGIN
- Participant-state: Introduced versioned form of rejection reason
CHANGELOG_END
* Upgrade protobuf-java and scalapb.
CHANGELOG_BEGIN
CHANGELOG_END
* Bazel: Generate ScalaPB classes without `unknownFields`.
This is very annoying because it means that extractors need an extra
`_` for no good reason. It also breaks Circe's automatic derivation.
Including unknown fields is the default behavior in Protocol Buffers for
Java now, and we'll probably have to reckon with it at some point, but
let's kick that can down the road a little.
* Upgrade protobuf-java to 3.17.1.
This is identical to 3.17.0, but let's track it anyway.
* Bazel: Move ScalaPB versions into their own file.
They don't need to go into the generated one.
* Damlification of Scala files (primarily comments and strings).
* Corrected a typo.
CHANGELOG_BEGIN
CHANGELOG_END
* Fixed build.
* Fixed test case for acronyms.
* Reformatted.
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
This PR includes :
- Adding ApiCommand to distinguish between generic command (that are
accepted by the engine) and command that are accepted by the ledger
API.
- Reimplement Canton's reinterpret method using commands instead of
node.
CHANGELOG_BEGIN
CHANGELOG_END
* Address more exception todos
A bit of a kitchen sink PR to address a bunch of the trivial todos
that didn’t seem worth splitting out into separate PRs.
changelog_begin
changelog_end
* Revert changes to TransactionSpec
changelog_begin
changelog_end
As discussed, we don’t want to expose this via serializable values at
least for now (and it’s not exposed on the ledger API anyway) so this
PR drops the type.
changelog_begin
changelog_end
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Change the TelemetryContext to use SpanAttributes instead of raw AttributeKeys
and remove an unused object along with its dependencies
CHANGELOG_BEGIN
CHANGELOG_END
* Change the Tracer name and the instrumentation name from participant to com.daml.telemetry
* Add telemetry classes from the oem integration kit and use it for command submissions
* Change submitTransaction to submitTransactionWithTelemetry and add a deprecation
* Fix tests
* Revert "Change submitTransaction to submitTransactionWithTelemetry and add a deprecation"
CHANGELOG_BEGIN
- [Integration Kit] TelemetryContext has been introduced to the WriteService.submitTransaction method to support distributed tracing
CHANGELOG_END
* Add new variant to Value.scala for builtin-exceptions.
final case class ValueBuiltinException[+Cid](tag: String, value: Value[Cid]) extends Value[Cid]
And push through the code consequences.
Most places fixed up.
A couple more things to do in this PR (marked NICK)
A couple of things which can be left for later (marked 8020)
fix build
fix another scala match
changelog_begin
changelog_end
* fix any match
* add marker of code which needs attending to in the PR
* extend ledger-api value.proto & fix LfEngineToApi
* undo/comment-out the change to value.proto
* add tests in HashSpec for BuiltinException
* code but dont yet enable value-gen for builtin exceptions
* address comments which suggest we crash in various places
* support BuiltinException in scenario_service.proto
* one more TODO 8020 tag
* Add test to verify that TLSv1.2 and TLSv1.3 work
As raised here: https://discuss.daml.com/t/what-version-of-tls-is-supported-by-the-ledger-api/1982
This has been effectively fixed by upgrading Netty here: https://github.com/digital-asset/daml/pull/8558
Note that this is an integration test for sandbox-classic only, but the
infrastructure used is shared with other Ledger API server
implementations, so this should be enough.
changelog_begin
changelog_end
* Fix Scala 2.13 compatibility issue
* Fix warning by adding type annotation to public field
* Port parts of //ledger/... to Scala 2.13
Fairly random choice of directories, I just went through them in
alphabetical order. The one thing that I had to disable for now are
the conformance tests since the ledger API test tool has a dependency
not compatible with Scala 2.13.
changelog_begin
changelog_end
* Remove accidentally included //ledger/ledger-api-client/...
doesn’t actually work yet
changelog_begin
changelog_end
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Port damlc dependencies to Scala 2.13
I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).
So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).
If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.
changelog_begin
changelog_end
* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* fixup lawlesstraversal
changelog_begin
changelog_end
* less iterator more view
changelog_begin
changelog_end
* document safety of unsafeWrapArray
changelog_begin
changelog_end
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* participant-integration-api: Dedicated execution context for requests.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Construct the services executor outside.
* participant-integration-api: Share the services EC with the GRPC stack.
* participant-integration-api: Use the new EC wherever possible.
And stop using DirectExecutionContext.
* sandbox-classic: Fix DevModeIT to use the right configuration.
I have no idea why this breaks now, but it was always technically
broken, so I'm fixing it.
* participant-integration-api: Push the services EC up one more level.
* Update newly added ApiParticipantPruning service accordingly
see https://github.com/digital-asset/daml/pull/7988#discussion_r525319097
* participant-integration-api: Ensure the LedgerConfigProvider is ready.
Somehow this slipped through.
* sandbox-classic: Fix `DefaultConfig` in tests.
* language-support/java: Disable seeding in tests again.
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
* Remove single-party check
CHANGELOG_BEGIN
- [Ledger API] The ledger API now supports multi-party submissions.
In order to use multi-party submissions, use the new act_as and
read_as fields in submission requests.
CHANGELOG_END
* Remove usage of temporary SubmitterInfo methods
* Fix validator tests
* Fix auth test
* Add multi-party tests to the ledger API
* Fix compile errors
* Improve tests
* Remove temporary single-party method from SubmitterInfo
* Remove temporary single-party method from SubmitterInfo companion object
* Remove temporary single-party method from TxEntry
* Run multi-party submission ITs for ledger-on-memory and ledger-on-sql
* Minor improvement
Co-authored-by: Kamil Bozek <kamil.bozek@digitalasset.com>
* Upgrade Scala dependencies for 2.13 compatibility
This upgrades a bunch of Scala libraries to versions that have 2.13
support. There are two libraries that are still missing:
- diffson, this has a new version but with significant breaking
changes and it is only used in Naigator console which I hope to kill
before I have to worry about this.
- ai.x:diff, this is used in the ledger API test tool. The library is
abondened but there are a few alternatives.
changelog_begin
changelog_end
* Fix pureconfig
changelog_begin
changelog_end
* Fix Navigator
changelog_begin
changelog_end
Changes:
- replaced submitter: Party with actAs: Set[Party] and readAs: Set[Party] in com.daml.ledger.api.domain.Commands
- adapted contract and key lookups for multi-party submissions
* Changed single Commands to accept multiple submitters
* Removed submitter field from Commands
* Replaced submitters with actAs and readAs in Commands
* Moved actAs and readAs to com.daml.ledger.api.Commands
* Contract lookup tests
* Key lookup tests
CHANGELOG_BEGIN
CHANGELOG_END
* Formatted code changes
* Remove unused method
* Formatting fixes
* Simplified logical condition for active contracts visibility
* Removed unused temporary method in Commands
* Unified contract key lookup query for different type of db types
* Simplified ContractReader class structure
* Improved dao unit tests
* Unit tests for lookups as a divulgee
* Reduced code duplication in unit tests
* Minor improvement
* Formatted code
* Fixed a ContractsReader's query
* Improved unit tests
* Explanation for engine's command authorizers and contract readers
* Minor fixes
* Adapted SpannedIndexService for the multi-party submission
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* Add multi-party submissions to the ledger API
CHANGELOG_BEGIN
* [Ledger API] Command submission requests now contain new optional
fields used for multi-party submissions. Such submissions currently
return UNIMPLEMENTED errors, they will be enabled in the future.
CHANGELOG_END
* Adapt Haskell bindings
* Handle the new ledger API fields
* Fix SubmitAndWaitDummyCommand
* Remove unused methods
* Redesign multi-party auth tests
* Remove direct access of request party
* Apply review comments
* Improve protobuf comments
* Multi-party tracker map
* Fix validation logic
* Consistent metric naming
* Multiple submitters field for Command
* Multiple submitters field for Engine API
* Tests for multi-party submissions in Engine
* Make use of multi-party Engine capabilities in the engine benchmark
* Enable multi-party submission in Engine's API
CHANGELOG_BEGIN
CHANGELOG_END
* Separate unit tests for multi-party engine
* Unit tests for multi-party submission transaction replay
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.
The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.
CHANGELOG_BEGIN
- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature
CHANGELOG_END
* Participant pruning ledger api server support ported from Canton
CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END
* Review feedback from Stefano
* Add pruning tests plus missed command completions change
* Review feedback from Robert
* Improved test readability by having populate helper return offsets
* Review feedback
* Ledger api changes to pruning api and disable canton pruning test
- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string
* Review feedback: Use ApiOffsetSConverter for logged offsets
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* add silent_annotations option to da scala bazel functions
* use silent_annotations for several scala targets
* use silencer_plugin instead when the lib isn't used
* use silent_annotations for several more scala targets
* use silencer_lib for strange indirect requirement for running tests
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* silent_annotations support for scaladoc
* Add metrics for concurrent commands
* Update readme
CHANGELOG_BEGIN
- [DAML on SQL] Add new metrics for measuring the number
of concurrent command executions. The metrics are:
daml.commands.submissions_running, daml.execution.total_running,
daml.execution.engine_running
CHANGELOG_END
* concurrent: Tag DirectExecutionContext.
1. Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
stating that it works for any tagged `Future`.
2. Move `DirectExecutionContext` to the _libs-scala/concurrent_
library, as it requires it and it's tiny.
CHANGELOG_BEGIN
CHANGELOG_END
* concurrent: Fix the privacy of `DirectExecutionContextInternal`.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* [KVL-222] Add participant id to index metadata dump
changelog_begin
changelog_end
* Test SqlLedger participant id initialization
* Test JdbcIndexer participant id initialization
* Make RecoveringIndexerSpec final and remove unused trait
* Refactor SQLLedger initialization routine
Small refactoring to make initialization a bit more readable. Performed
while moving forward with the addition of the participant identifier to
the parameters table (so a few minor details have leaked into this PR).
changelog_begin
changelog_end
* Fix compilation errors
* Address https://github.com/digital-asset/daml/pull/7200#discussion_r474630880
* Fix test, lower test logging noise
* remove unused definitions, params, args from ledger API Scala code
CHANGELOG_BEGIN
- [Ledger API] withTimeProvider removed from CommandClient; this method
has done nothing since the new ledger time model was introduced in
1.0.0. See `issue #6985 <https://github.com/digital-asset/daml/pull/6985>`__.
CHANGELOG_END
* percolate withTimeProvider and label removal elsewhere
* Change error code for invalid offsets for transaction stream and completion stream requests
* Expanded application architecture docs on how to build application with ledger api failover capabilities.
Fixes#6842.
CHANGELOG_BEGIN
- [Ledger API] The error code for requesting a transaction stream
with an offset beyond the ledger end changed from INVALID_ARGUMENT
to OUT_OF_RANGE. This makes it easier to handle scenarios where
an application fails over to a backup participant which hasn't
caught up with the ledger yet.
- [Ledger API] The command completion service now validates the offset and
returns the OUT_OF_RANGE error if the request offset is beyond the ledger end.
- [Documentation] Added a section on how to write DAML applications
that can fail over between multiple eventually consistent Ledger API endpoints
where command deduplication works across these Ledger API endpoints, which
can be useful for addressing HA and/or DR scenarios.
CHANGELOG_END
Also fixes#5635, removing usage of `ReadService` from the `StandaloneApiServer`.
The configuration stream of the Ledger API LedgerConfigurationService is now properly backed by the configuration entries instead of just serving the initial configuration.
CHANGELOG_BEGIN
[DAML Integration Kit]: ``StandaloneApiServer`` can now be run in a read-only mode.
- The type of the constructor parameter ``writeService`` of ``StandaloneApiServer`` changed to ``Option[WriteService]``. Passing ``None`` will not start any of the admin services, the command service, and the command submission service.
- The constructor parameter ``readService`` of ``StandaloneApiServer`` has been removed.
- A new constructor parameter ``ledgerId`` has been added to ``StandaloneApiServer``. It is used to verify that that ``StandaloneApiServer`` is run against an index storage for the same ledgerId. Initialization is aborted if this is not the case.
[DAML Integration Kit]: The ``LedgerConfigurationService`` now properly streams configuration changes.
CHANGELOG_END
* replace traverseU and sequenceU with traverse and sequence
- with -Ypartial-unification on, the extra Unapply typeclass lookup is
unnecessary
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* limit imports; we only need *> and void
