To enable direct IndexServer building (and enable two step ApiServer build process)
This will be needed for KV WriteService proxy: a wrapper is being build to serve deduplication time conversion, which needs access to completion service.
changelog_begin
changelog_end
kvutils - The deduplication period is no longer overwritten by the max deduplication duration, therefore there is no need to change the default max deduplication duration
changelog_begin
changelog_end
It never did anything in Sandbox Next. Packages are always loaded
eagerly in KV ledgers.
CHANGELOG_BEGIN
- [Sandbox] The ``--eager-package-loading`` flag has been deprecated. It
hasn't actually done anything for many releases; packages are always
loaded eagerly. This does not affect Daml on SQL, which does support
lazy package loading.
CHANGELOG_END
* Remove the mutating schema
changelog_begin
- [Participant] All participants now use the new append-only schema. Existing databases will
automatically upgrade to the new schema the first time a participant/ledger is started.
changelog_end
* Fix post-commit validation test
* Remove append-only flags from CI
* Don't crash when using deprecated flag
* Increase timeout for reset service tests
* Fix typos in parameter names
* Restore removed test
* Restore removed CLI check
* Improve CLI parameter description
* [Self-service error codes] Implement V2 in Authorizer
CHANGELOG_BEGIN
CHANGELOG_END
* Added unit test for authorize (non-streamed)
* Fix after rebase
* Do not expose the error codes switching mechanism to the Java bindings
* Adjust InternalAuthorizationError to be SystemInternalAssumptionViolated
* Parameter names in test
* Testing AuthorizationInterceptor with regard to returned error codes
* Do not use default error code version switchers at instance creation
* Addressed Pawel's review comments
* Using ErrorFactories for error dispatching
* Pass loggingContext to Authorizer where available
* Generic internal authorization error
* ledger-api-test-tool: Add command line argument --static-time which makes the test tool aware that the ledger is running in static time
For command deduplication conformance tests this allows us to forward the time so that the tests can run in static time mode as well
CHANGELOG_BEGIN
CHANGELOG_END
* Update ledger/ledger-api-test-tool/src/main/scala/com/daml/ledger/api/testtool/Cli.scala
* Do not run conformance test that cannot run in static time
* Remove hidden from static time flag for ledger api test tool
* ledger-api-test-tool: Clarify parts of the deduplication tests.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
* Use append-only schema in tests
changelog_begin
changelog_end
* Fix Oracle build
* Update ledger/ledger-on-memory/BUILD.bazel
Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
* Run more conformance tests on append-only
* Run pruning tests in split-participant mode
* Disable multi-participant test in split participant
* Add missing ContractIdIT test
Co-authored-by: tudor-da <tudor.voicu@digitalasset.com>
* Extract common code for command dedup conformance tests
CHANGELOG_BEGIN
CHANGELOG_END
* Revert changes for parallel tests
* Fix compile warnings
* Disable participant deduplication for KV
CHANGELOG_BEGIN
CHANGELOG_END
* Always call submitRequestAndAssertDeduplication with parties
* Fix party in CommandDeduplicationBase
* Don't test KV-based command deduplication with static time as it's broken
* Don't disable participant-side command deduplication for sandbox-next on static time
* Revert "Don't test KV-based command deduplication with static time as it's broken"
This reverts commit 3dac41aa
* Add missing dependency
* Don't cross-test this ledger API test tool with existing versions
* Add the current PR to the list of PRs that resulted in exclusions
* Apply @garyverhaegen-da's suggestion for compatibility exclusions
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Fix broken test and use consistency for tests
* ledger-api-test-tool - Add conformance test for parallel command deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Add import for 2.12 compat
* Add silencer plugin
* Split parallel command deduplication scenario into it's own test suite
* Add the parallel command deduplication test to append only ledgers
* Run parallel command deduplication tests for append only ledgers
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Code review renames
* Add compat import
* Run the test concurrently
* Support adding tests as an hidden option
* Simplify existing suites
CHANGELOG_BEGIN
CHANGELOG_END
* Remove stale conformance suites from build.yml
* `--add` -> ``--additional`
* Re-add `--all-tests` as deprecated CLI option to be tested
* Move sandbox-classic pruning test to wall clock again
* Run KVCommandDeduplicationIT for sandbox append-only
* Tidy-up
* Also add participant pruning test to ledger-on-memory/single-participant
* Remove KVCommandDeduplicationIT on ledger-on-memory/append-only
* Run the full suite plus pruning (rather than just pruning) for ledger-on-memory with multiple participants and append-only
* Add KVCommandDeduplicationIT to ledger-on-memory append-only
* Exclude ConfigManagementServiceIT from ledger-on-memory append-only multi-participant
* Tidy-up
* Use KVCommandDeduplicationIT for sandbox-on-x too
* Fix merge
Add max dedup duration arg to all the test suites that include command dedup tests
* Make `--include` and `--additional` mutually exclusive
* Uniform formatting of multi-line strings
* Move exclusions after additions as they are applied last
* Re-disable deduplication test on sandbox with static time
* Re-disable deduplication test on sandbox-on-x
* Add conformance test for KV committer usage of max deduplication duration as deduplication duration
CHANGELOG_BEGIN
CHANGELOG_END
* Run AppendOnlyKVCommandDeduplicationIT for KV ledgers that use the append-only schema
* Always use max_deduplication_duration as deduplication period for committer side deduplication
Doing this we get the following guarantees:
- Deduplication period is constant for all the submissions therefore this guarantees consistency between "backward-looking" and "forward-looking" deduplication
CHANGELOG_BEGIN
kvutils - committer side deduplication always uses max_deduplication_duration + min_skew as a deduplication period for all the requests.
CHANGELOG_END
* Set max_deduplication_duration to 10 seconds for all the conformance tests
* Update KV command deduplication conformance tests to account for max deduplication duration used as deduplication period for all the requests
* Add max deduplication wait for multi-participant conformance test
* Overwrite the submitter info deduplication period in the transaction committer so that we always set the max deduplication duration
* Update ledger/participant-state/kvutils/src/test/suite/scala/com/daml/ledger/participant/state/kvutils/committer/transaction/TransactionCommitterSpec.scala
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Add separate step for overwriting the deduplication period
* Code review - reformatting and renames
* Add comment for setting the deduplication duration
* Add Ledger API Test Tool tests about command deduplication information present in completions
CHANGELOG_BEGIN
CHANGELOG_END
* Address review comments
* Fix offset reference and move new tests to a separate, optional suite
* Cover rejections as well
* Test both the command and command submission services
* Run new test suite (only) on sandbox-classic append-only and daml-on-sql
* Use the append-only schema with daml-on-sql for CommandDeduplicationInfoIT
* Fix tests except offset, successful completions only.
* Remove completion offset test as it's not supported by most ledgers
* Remove support for multiple submissions as rejection completions are not being tested for the moment
* Consolidate test cases for faster run
* Avoid forbidden characters in short identifiers
* Clarify assert
* Remove wrong test about deduplication time being preserved in completion
* Adhere to the Scala style guide
* Eliminate some code duplication
* fmt
* Make 3e404be compile
* Never assume a specific deduplication period format in completions
* Code cleanup
* Pass party to completionStreamRequest to avoid ILLEGAL_ARG failures
* Enable in append-only mode for all ledgers that support it
* Clarify that the new tests are append-only-only
* Update ledger/ledger-on-sql/BUILD.bazel
Fix suite name
* Update ledger/sandbox-classic/BUILD.bazel
Fix suite name
* fmt
* Split command deduplication conformance tests into non-kv and kv tests.
Because kv ledgers have both participant and committer side deduplication, the conformance tests have been duplicated into different suited which can target either both deduplications or just participant side deduplication.
CommandDeduplicationIT is a default suite and should be run for ledgers with participant side only deduplication.
KVCommandDeduplicationIT is optional and should be enabled for any KV ledgers.
CHANGELOG_BEGIN
ledger-api-test-tool - CommandDeduplicationIT tests only participant side deduplication, it should be disabled for KV ledgers. Added KVCommandDeduplicationIT which should be run for KV ledgers
CHANGELOG_END
* Run participant side/committer side deduplication tests based on the ledger implementation
* Fix on memory build file
* Daml-on-sql is not a kv ledger so revert the changes so that it runs the default command deduplication IT
* Rollback compat import
* Exclude command dedup from the reference test
* Exclude command deduplication from the reference export
* Format bazel
* Set the exclude to client args for the ledger-api-test-tool and not for the ledger-on-memory
* Exclude CommandDeduplicationIT from non repudiation conformance tests and include KVCommandDeduplicationIT because it runs using sandbox
* Exclude HealthServiceIT from the reference-ledger-export as it has no impact on the export
* Exclude CommandDeduplicationIT from the ledger-on-sql conformance-test as they are KV ledgers
* Format bazel file
* Exclude CommandDeduplicationIT from sandbox
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Add comments for command deduplication IT
* Rename to be consistent with scala namings
* Update formatting and comments
* Add naming prefix for tests to avoid clashes
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Enable append-only schema in sandbox
changelog_begin
changelog_end
* Add test
* Unhide the compression flag in sandbox
* Make append-only config non-optional
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* participant-integration-api: Encapsulate the initial configuration.
* participant-integration-api: Reduce usage of `LedgerConfiguration`.
* Inline `LedgerConfiguration` wherever it's used.
Most things don't need all its constituent parts; this reduces the
amount of unused properties.
CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
removed in favor of ``InitialLedgerConfiguration``. Its usage
has been changed accordingly, with the ``configurationLoadTimeout``
property becoming part of ``ApiServerConfig`` instead.
The default options provided by ``LedgerConfiguration`` have been
removed; you are now encouraged to come up with sensible values for
your own ledger. The ``Configuration.reasonableInitialConfiguration``
value may help.
CHANGELOG_END
* Correct the initial configuration submission delay for KV ledgers.
* kvutils: Mark supertype unused parameters as unused.
* kvutils: Extract out common configuration submission delays.
These values are specific to kvutils; other drivers should come up with
their own.
* configuration: Delete `NoGeneration`, as it's unused.
The conversion of SValue to Value already ensures the resulting value
has a serializable type. Here we add a check to ensure it does not
overpass the maximum allow nesting.
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
* ledger-offset: Move `Offset` to a new package.
CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
CHANGELOG_END
* Avoid rewrapping offsets for no reason.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-integration-api: Sort some imports.
* participant-integration-api: Fix dependencies for the Oracle tests.
I didn't add `ledger-offset`.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-configuration: Extract configuration from participant-state.
The configuration is often used without the state, and doesn't need to
be versioned in the same way.
CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
CHANGELOG_END
* participant-state: Remove the `LedgerId` aliases.
* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.
This avoids confusion with the protobuf-generated `TimeModel` classes.
CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
CHANGELOG_END
* ledger-configuration: Remove colons in LedgerInitialConditions' docs.
* kvutils: Restore a missing compat import.
* participant-integration-api: Add ledger-configuration to Oracle tests.
* sandbox-common: Fix `--max-ledger-time-skew` docs.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Move Daml Profiler to EE version of sandbox/sandbox-classic
This splits Sandbox targets into EE/CE targets and exposes the option
in the EE version. The option still exists in the CE option for now
until we have released EE artifacts to not break users that might know
about it without an alternative.
There is also a small test that makes sure that this actually works
since classpaths are dumb and it didn’t work at first.
changelog_begin
changelog_end
* Fix publish target
changelog_begin
changelog_end
* Publish transitive dep
changelog_begin
changelog_end
* I hate bash
changelog_begin
changelog_end
* Ledger API test tool: test against legacy and preview version.
+ use dictionary instead of alias to map version keywords to LF version
CHANGELOG_BEGIN
CHANGELOG_END
improve previous generalization from #8695
- use lf version instead keyword (like 'stable', 'latest', 'dev') to
tag actual target. This will allow two keywords to map to the same
versions without doing twice the compilation/test work.
- use alias to map keywords tag target to versioned tag target.
- move package manage dar to test_commong.
CHANGELOG_BEGIN
CHANGELOG_END
* Port the rest of //ledger/... to Scala 2.13
draw the rest of the fcking owl
Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).
changelog_begin
changelog_end
* Less symbols
changelog_begin
changelog_end
* Port //ledger/ledger-api-client/... to Scala 2.13
This pulls in Sandbox next and kvutils as a dependency so those now
build on 2.13 as well.
changelog_begin
changelog_end
* Upgrade scala-colllection-compat
changelog_begin
changelog_end
* Use toVector.sortBy instead of to(LazyList).sortBy
changelog_begin
changelog_end
* Use a view for passing things to varargs
changelog_begin
changelog_end
* avoid symbol literal in CommandClientIT
changelog_begin
changelog_end
The flag allows the sandboxes (both classic and next) to use preview
version of the upcoming Daml-LF version (currently LF 1.11).
CHANGELOG_BEGIN
* [Sandbox-classic] add a flag to allow early access to the next
Daml-LF to be released.
* [Sandbox] add a flag to allow early access to the next Daml-LF
to be released.
CHANGELOG_END
* kvutils: Use ScalaPB to generate a Scala JAR for daml_kvutils.proto.
* Bazel: Delete the unused `da_java_binary` rule, and inline `_wrap_rule`.
* Bazel: Factor out Java/Scala protobuf class generation into a helper.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/archive: Use `proto_jars`.
* Bazel: Remove the visibility modifier from `proto_jars`.
It's too confusing. Just make everything public.
* daml-lf/archive: Push protobuf source tarballs into `proto_jars`.
* Bazel: Add comments to the various parts of `proto_jars`.
* daml-assistant: Do unpleasant things with `location` in Bazel.
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.
The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.
CHANGELOG_BEGIN
- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature
CHANGELOG_END
* Suppress debug and trace logging for Sandbox tests.
* participant-integration-api: Don't submit config after shutting down.
* sandbox: When testing the reset service, wait for the ledger config.
If we don't wait for the ledger configuration, we could potentially ask
for completions before the server is fully ready.
* sandbox-common: Reduce the number of reset attempts in testing.
Now we're also checking for ledger configuration, the time for a reset
to be "accepted" is longer, so 5 tries in 30 seconds is a bit flaky.
Reducing to 4 seems reasonable.
* sandbox: Stop marking the reset service tests as flaky.
I don't have conclusive proof that they're no longer flaky, but I'm
fairly confident. We can revert this commit if they turn out to still
need some attention.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-common: Check the configuration shows up in reset service tests.
* sandbox-common: Factor out logback-test.xml.
* resources: Move builders into //ledger/ledger-resources.
Keep the actual constructors in a trait, but instantiate it when working
with ledger code.
This allows us to later introduce an extra "context" type parameter to
ResourceOwner.
* resources-akka: Move the builders in to //ledger/ledger-resources.
* resources: Introduce an abstract `Context` parameter for owners.
This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.
This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.
Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.
CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
`Context`, which is filled in by the corresponding `Context` class in
the _ledger-resources_ dependency. This allows us to pass extra
information through resource acquisition.
CHANGELOG_END
* ledger-resources: Move `ResourceOwner` here from `resources`.
* ledger-resources: Remove dependencies from outside //ledger.
* ledger-resource: Wrap the acquisition execution context in `Context`.
So we can add a logging context to it.
* resources: Pass the Context, not the ExecutionContext, to Resource.
* Avoid importing `HasExecutionContext`.
* ledger-resources: Publish to Maven Central.
* resources: Make the small changes suggested by @stefanobaghino-da.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* ledger-resources: Pull out a trait for test resource contexts.
Saves a few lines of code.
* Restore some imports that were accidentally wildcarded.
* resources: Replace an `implicit def` with a couple of imports.
* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.
Try and use the right execution context where possible.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
