* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* sandbox: Do less when resetting.
We don't need to reconstruct some basic things.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: When testing the reset service, keep trying.
If we scale up the test duration, we also need to scale up the timeouts.
* timer-utils: If `RetryStrategy` eventually fails, explain what happened.
For example, if the attempts limit is exceeded, say so.
* Revert "timer-utils: If `RetryStrategy` eventually fails, explain what happened."
This reverts commit 6647681688.
* participant-integration-api: Make the initial ledger config optional.
CHANGELOG_BEGIN
- [Integration Kit] The initial ledger configuration is now optional;
if it is not specified, the participant will not attempt to submit a
configuration, but instead wait for something else to provision the
ledger with a configuration.
This behavior is intended to be used by ledger drivers that have
alternative means of setting up the initial ledger configuration. If
no configuration is provisioned, the participant server will not be
able to submit commands to the ledger.
CHANGELOG_END
* participant-integration-api: Avoid `Option#zip`.
It returns an `Iterable` in Scala 2.12, which is not helpful.
* participant-integration-api: Encapsulate the initial configuration.
* participant-integration-api: Reduce usage of `LedgerConfiguration`.
* Inline `LedgerConfiguration` wherever it's used.
Most things don't need all its constituent parts; this reduces the
amount of unused properties.
CHANGELOG_BEGIN
- [Integration Kit] The ``LedgerConfiguration`` class has been
removed in favor of ``InitialLedgerConfiguration``. Its usage
has been changed accordingly, with the ``configurationLoadTimeout``
property becoming part of ``ApiServerConfig`` instead.
The default options provided by ``LedgerConfiguration`` have been
removed; you are now encouraged to come up with sensible values for
your own ledger. The ``Configuration.reasonableInitialConfiguration``
value may help.
CHANGELOG_END
* Correct the initial configuration submission delay for KV ledgers.
* kvutils: Mark supertype unused parameters as unused.
* kvutils: Extract out common configuration submission delays.
These values are specific to kvutils; other drivers should come up with
their own.
* configuration: Delete `NoGeneration`, as it's unused.
* participant-integration-api: Move test constants to a companion object.
TelemetrySpecBase doesn't need to expose those through inheritance.
* Avoid mocks in the configuration management service test.
* Add tests for `ConfigManagementService.GetTimeModel`.
* Add tests for `ConfigManagementService.SetTimeModel`.
* Disable configuration management until the ledger has a configuration.
CHANGELOG_BEGIN
- [API Server] The configuration management service previously returned
the participant-specified default configuration if none was found on
the ledger. This can be misleading, and so the ``GetTimeModel``
endpoint now returns a gRPC ``NOT_FOUND`` error if no configuration
has been found yet. Similarly, the ``SetTimeModel`` endpoint returns a
gRPC ``UNAVAILABLE`` error.
This should only happen if the participant gives up waiting for the
ledger to provide a configuration, which is very unlikely in a
production setting. In this case, the ledger will also not respond to
command submissions.
CHANGELOG_END
* participant-integration-api: Extract some test variables for clarity.
In ApiConfigManagementServiceSpec.
* sandbox: Improve some variable and parameter names in tests for clarity.
* Improve the error message in case of a missing ledger configuration.
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* ledger-api-common: Improve an error status constructor name.
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* participant-state: Give `ChangeId` its own file.
* participant-state: Introduce `v1.CompletionInfo` for a while.
`CompletionInfo` is identical to `SubmitterInfo`. Its purpose is to make
the migration to v2 easier. It should not make it into the final
version.
* participant-integration-api: Switch to the v2 participant state API.
This means that:
- the API server and indexer expect v2 API traits
- adapters are provided if you need to elevate your v1 API usage
- the indexer internally uses v2 Updates
- rejections are switched over to use the v2 format
- Sandbox Classic uses v2 as the internal representation too (because
it interacts directly with the underlying indexer representation,
and is therefore tightly coupled)
_kvutils_ and other users of the `StandaloneApiServer` and
`StandaloneIndexerServer` use the adapters.
CHANGELOG_BEGIN
- [Integration Kit] The API server and indexer have switched over to v2
of the participant-state API. You can continue to use the v1 API, but
you will need to wrap your ``ReadService`` and ``WriteService``
objects in the ``AdaptedV1ReadService`` and ``AdaptedV1WriteSerivce``
classes.
CHANGELOG_END
* participant-state: Remove v1.CompletionInfo.
It's served its purpose.
* kvutils: Remove an unnecessary line from `Runner`.
* ledger-api-common: Delete a TODO; we'll track it elsewhere.
* participant-integration-api: Use full words in `JdbcLedgerDao`.
Just a little bit of cleanup.
* ledger-api-common: Extract out the random submission ID generator.
And introduce a trait, because, well, this is still the JVM.
The conversion of SValue to Value already ensures the resulting value
has a serializable type. Here we add a check to ensure it does not
overpass the maximum allow nesting.
* participant-state: Remove the aliases to `Ref.LedgerString`.
Use the aliases directly in `Ref` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/data: Improve comments on ID aliases.
* daml-lf/data: Fix a typo in a comment.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
* ledger-offset: Move `Offset` to a new package.
CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
CHANGELOG_END
* Avoid rewrapping offsets for no reason.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-integration-api: Sort some imports.
* participant-integration-api: Fix dependencies for the Oracle tests.
I didn't add `ledger-offset`.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-configuration: Extract configuration from participant-state.
The configuration is often used without the state, and doesn't need to
be versioned in the same way.
CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
CHANGELOG_END
* participant-state: Remove the `LedgerId` aliases.
* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.
This avoids confusion with the protobuf-generated `TimeModel` classes.
CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
CHANGELOG_END
* ledger-configuration: Remove colons in LedgerInitialConditions' docs.
* kvutils: Restore a missing compat import.
* participant-integration-api: Add ledger-configuration to Oracle tests.
* sandbox-common: Fix `--max-ledger-time-skew` docs.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
I don't see a reason why it's part of the participant state API, and
it definitely doesn't need to change between v1 and v2.
CHANGELOG_BEGIN
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
CHANGELOG_END
* ledger-on-sql: Abstract out the log entry ID allocator.
This is so we don't need to depend on `SeedService`.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Remove `allocateRandomLogEntryId` from production code.
* kvutils: Remove `SubmissionValidator.create`, as it's only for tests.
And remove default parameters for
`SubmissionValidator.createForTimeMode`.
* ledger-on-sql: Use the random log entry ID allocator.
In Sandbox, we still use the seed service, but this is not necessary
for ledger-on-sql tests.
* kvutils: Make `LogEntryIdAllocator.random` an `object`.
* ledger-on-sql: Move `RandomLogEntryIdAllocator` here.
It's not used anywhere else.
* ledger-on-sql: Add some documentation about the application code.
Just to make it clear it's not used in production.
* Adds Indexer state to GRPC health checks [DPP-434]
CHANGELOG_BEGIN
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
CHANGELOG_END
* Addressed review comments
* Introduce metrics in the http-json service
changelog_begin
[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)
changelog_end
* Move MetricsReporter and it's dependencies into //ledger/metrics
* Restore non-ugly formatting for that one section in Endpoints.scala
* Update ledger/sandbox-common/src/test/lib/scala/platform/sandbox/cli/CommonCliSpecBase.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Hide metrics option for http-json
* Propagate context exception for the parseUri function in MetricsReporter.scala
* Move cliHint value above parseUri function to have a better structure (it's used once before it's defined and once after it's defined, which is weird to me)
* Use better value name than optMr & optFd in cliopts Metrics.scala
* Remove import order changes & whitespace changes
* Revert usage of Nanoseconds for conversion from scala duration to java duration to usage of Seconds
* Shorten hideIfRequested function
* Fix another rearranged import
* Fix another whitespace removal
* Readd metrics cli option to sandbox after refactoring
* Add missing return type annotation for invalidRead in MetricsReporter
* Readd newline in https OptionParser.scala
* Remove unecessary import
* Update ledger/sandbox-common/src/main/scala/platform/sandbox/cli/CommonCliBase.scala
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Align setter & config name for metricsReportingInterval setting too in CommonCliBase.scala
* Rename http_json_api in Metrics.scala of metrics project to HttpJsonApi
* Reformat CommonCliBase.scala of sandbox-common project
* Fix CommonCliSpecBase test of sandbox
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* [In-memory fan-out] BuffersUpdater implementation
* Instantiated the TransactionsBuffer in ReadOnlySqlLedgerWithMutableCache
* Implementations and tests for the BuffersUpdater which updates:
* * The MutableCacheBackedContractStore
* * TransactionsBuffer
* Introduced `buffered-ledger-api-streams-unsafe` CLI flag for enabling in-memory fan-out flow
* Enabled the `buffered-ledger-api-streams-unsafe` in ledger-on-sql conformance test for append-only schema
CHANGELOG_BEGIN
CHANGELOG_END
* [In-memory fan-out] BuffersUpdater implementation
* Instantiated the TransactionsBuffer in ReadOnlySqlLedgerWithMutableCache
* Implementations and tests for the BuffersUpdater which updates:
* * The MutableCacheBackedContractStore
* * TransactionsBuffer
* Introduced `buffered-ledger-api-streams-unsafe` CLI flag for enabling in-memory fan-out flow
* Enabled the `buffered-ledger-api-streams-unsafe` in ledger-on-sql conformance test for append-only schema
CHANGELOG_BEGIN
CHANGELOG_END
* 2 seconds rather than 250ms for default hikari connection pool timeout
CHANGELOG_BEGIN
[jdbc ledger] increase default hikari connection pool timeout to 2s from 250ms
CHANGELOG_END
* extract hikari connection timeout configuration
* default of 250ms
* read hikari connection timeout default from config rather than hardcoding for sandbox next
* remove unused import
* Damlification of Scala files (primarily comments and strings).
* Corrected a typo.
CHANGELOG_BEGIN
CHANGELOG_END
* Fixed build.
* Fixed test case for acronyms.
* Reformatted.
* Enable append-only schema in index
changelog_begin
changelog_end
* Enable append-only schema in indexer
* Add CLI flags to configure append-only indexer
* Fix CLI flag name
* Remove unused parameter
* Change CLI flag description
... it's independent of Postgres
* Refactor how indexer config is specified in CLI
* Upper case constants
* Move Daml Profiler to EE version of sandbox/sandbox-classic
This splits Sandbox targets into EE/CE targets and exposes the option
in the EE version. The option still exists in the CE option for now
until we have released EE artifacts to not break users that might know
about it without an alternative.
There is also a small test that makes sure that this actually works
since classpaths are dumb and it didn’t work at first.
changelog_begin
changelog_end
* Fix publish target
changelog_begin
changelog_end
* Publish transitive dep
changelog_begin
changelog_end
* I hate bash
changelog_begin
changelog_end
* Ledger API test tool: test against legacy and preview version.
+ use dictionary instead of alias to map version keywords to LF version
CHANGELOG_BEGIN
CHANGELOG_END
improve previous generalization from #8695
- use lf version instead keyword (like 'stable', 'latest', 'dev') to
tag actual target. This will allow two keywords to map to the same
versions without doing twice the compilation/test work.
- use alias to map keywords tag target to versioned tag target.
- move package manage dar to test_commong.
CHANGELOG_BEGIN
CHANGELOG_END
[ledger-api-server] Make database connection pool size configurable
CHANGELOG_BEGIN
[Daml Driver for PostgreSQL]: Added CLI option --database-connection-pool-size
to configure the size of the database connection pool
[Integration Kit]: Added the CLI options api-server-connection-pool-size and
indexer-server-connection-pool-size to configure the database connection pool size
for the Ledger API Server and the indexer respectively.
CHANGELOG_END
* Add CLI options to enable daml-lf dev version
* Compile test-common with multiple compiler versions
* Run the ledger API test tool with multiple compiler versions
changelog_begin
changelog_end
* Fix scaladoc issues
* Apply reviewer comments
regarding lf_preview_version
* participant-integration-api: Reuse the services EC for data munging.
Rather than spawning a separate thread pool for manipulating data before
and after index queries, we can just re-use the services EC, which is
already a work-stealing thread pool.
* kvutils: Share the services EC with the indexer.
CHANGELOG_BEGIN
CHANGELOG_END
* Removed logging JDBC url to prevent leaking passwords to log files.
CHANGELOG_BEGIN
- not logging JDBC url
CHANGELOG_END
* Minor change to fix the build
* Port the rest of //ledger/... to Scala 2.13
draw the rest of the fcking owl
Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).
changelog_begin
changelog_end
* Less symbols
changelog_begin
changelog_end
* Port //ledger/ledger-api-client/... to Scala 2.13
This pulls in Sandbox next and kvutils as a dependency so those now
build on 2.13 as well.
changelog_begin
changelog_end
* Upgrade scala-colllection-compat
changelog_begin
changelog_end
* Use toVector.sortBy instead of to(LazyList).sortBy
changelog_begin
changelog_end
* Use a view for passing things to varargs
changelog_begin
changelog_end
* avoid symbol literal in CommandClientIT
changelog_begin
changelog_end
The flag allows the sandboxes (both classic and next) to use preview
version of the upcoming Daml-LF version (currently LF 1.11).
CHANGELOG_BEGIN
* [Sandbox-classic] add a flag to allow early access to the next
Daml-LF to be released.
* [Sandbox] add a flag to allow early access to the next Daml-LF
to be released.
CHANGELOG_END
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
This commit fixes a few copyright headers that have been missed in the
automatic update on Jan 1, as well as the generation code in the compat
workspace so it generates the right headers.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Dedicated execution context for requests.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Construct the services executor outside.
* participant-integration-api: Share the services EC with the GRPC stack.
* participant-integration-api: Use the new EC wherever possible.
And stop using DirectExecutionContext.
* sandbox-classic: Fix DevModeIT to use the right configuration.
I have no idea why this breaks now, but it was always technically
broken, so I'm fixing it.
* participant-integration-api: Push the services EC up one more level.
* Update newly added ApiParticipantPruning service accordingly
see https://github.com/digital-asset/daml/pull/7988#discussion_r525319097
* participant-integration-api: Ensure the LedgerConfigProvider is ready.
Somehow this slipped through.
* sandbox-classic: Fix `DefaultConfig` in tests.
* language-support/java: Disable seeding in tests again.
Co-authored-by: Oliver Seeliger <oliver.seeliger@digitalasset.com>
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
