* Move public code into daml-integration-api
CHANGELOG_BEGIN
[DAML Integration Kit]: Removed sandbox specific code from the API intended to be used by ledger integrations. Use the maven coordinates ``com.daml:participant-integration-api:VERSION`` instead of ``com.daml:ledger-api-server`` or ``com.daml:sandbox``.
CHANGELOG_END
Also fixes#5635, removing usage of `ReadService` from the `StandaloneApiServer`.
The configuration stream of the Ledger API LedgerConfigurationService is now properly backed by the configuration entries instead of just serving the initial configuration.
CHANGELOG_BEGIN
[DAML Integration Kit]: ``StandaloneApiServer`` can now be run in a read-only mode.
- The type of the constructor parameter ``writeService`` of ``StandaloneApiServer`` changed to ``Option[WriteService]``. Passing ``None`` will not start any of the admin services, the command service, and the command submission service.
- The constructor parameter ``readService`` of ``StandaloneApiServer`` has been removed.
- A new constructor parameter ``ledgerId`` has been added to ``StandaloneApiServer``. It is used to verify that that ``StandaloneApiServer`` is run against an index storage for the same ledgerId. Initialization is aborted if this is not the case.
[DAML Integration Kit]: The ``LedgerConfigurationService`` now properly streams configuration changes.
CHANGELOG_END
* caching: Split caches into new files.
* caching: Rename `Cache.from` to `WeightedCache.from`.
* caching: Move `Configuration` inside `WeightedCache`.
* caching: Add test cases.
* caching: Allow for Caffeine builders to be covariant.
* caching: When instrumenting the Caffeine cache, compose, don't inherit.
* caching: Add a size-based cache.
* caching: Extract out common test cases into base classes.
* caching: Use the size-based cache for LF value translation.
CHANGELOG_BEGIN
CHANGELOG_END
* caching: Simplify the eviction tests.
* caching: Increase the encapsulation in CaffeineCache.
* caching: Commas are important.
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Extract caching from participant-state as a library
This will be used to keep a cache of values to cut on LF translation cost when serving transactions.
changelog_begin
changelog_end
* Add dependency where missing
CHANGELOG_BEGIN
- [Sandbox] The ledger API server will now always use the most recent ledger configuration.
Until a ledger configuration is read from the ledger, command submissions will fail with the UNAVAILABLE error.
CHANGELOG_END
In kvutils, the first ledger configuration change needs
to have a generation one higher than the one returned
by getLedgerInitialConditions().
Remove initial config writing from sandbox as it's now written by the ledger API server
* Sandbox: expose back pressure config in CLI
CHANGELOG_BEGIN
[Sandbox]: Added ``--max-commands-in-flight`` as CLI configs. See ``daml sandbox --help``.
[Sandbox Classic]: Added ``--max-commands-in-flight`` and
``--max-parallel-submissions`` as CLI configs. See ``daml sandbox-classic --help``.
CHANGELOG_END
* Bumping the default maxParallelSubmissions to 512 for sandbox classic
We used to use `maxCommandsInFlight * 2` in SqlServer, but it makes more
sense to use `maxParallelSubmissions` there. Since the lower default value of 128
would result in the conformance tests to fail, I'm bumping it to 512
* Use maxCommandsInFlight to configure the parallel submissions for CommandService
- Only upload packages during the initial startup.
- Avoid loading packages during subsequent resets
- Share an engine between Ledger API Server and Committer
* Use a randomized H2 URL to simular in-memory
The reset service test assumes to get a completely new ledger for each
test case. But because we use H2 in-memory with db_close_delay=1 and the
same H2 database name, the second test case gets the remnants of the
first test case.
Since we know that sandbox in-memory uses an H2 in-memory URL, we can
simply use SandboxBackend.H2Database for ResetServiceInMemoryIT.
CHANGELOG_BEGIN
[Sandbox] Drastically lower the time needed to do a reset via the
ResetService.
CHANGELOG_END
In #5419 we set specific loggers to WARN or ERROR so that we avoid noisy
logging by default. However, if the user specifies the --log-level CLI
param, not only the root logger, but also other loggers set in logback.xml
should log at that level.
CHANGELOG_BEGIN
[Sandbox] Properly respect the --log-level CLI parameter
CHANGELOG_END
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
Turns out not every participant can support seeding (yet).
CHANGELOG_BEGIN
- [Ledger API Server] Re-introduce an option to disable seeding. This
does not affect Sandbox.
CHANGELOG_END
* kvutils: Cache state value conversions from bytes.
This seems to have a decent speedup in ledger-on-memory.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Submissions now look up ledger values from a
cache where possible, improving performance when there's contention over
certain resources (e.g. common packages). The cache size currently
defaults to 64 MB.
CHANGELOG_END
* kvutils: Make the SubmissionValidator statue value cache configurable.
* kvutils: Report state value cache metrics.
* kvutils: Add a suffix to a Long literal because WartRemover is unhappy.
Strangely, it doesn't fail on my machine.
* kvutils: Extract caching out into its own file.
* kvutils: Move the `bytesToStateValue` call into `cache.get`.
* kvutils: Move caching to its own package.
* kvutils: Inject the state value cache.
* kvutils: Default to no state value cache.
* kvutils: Accept a state value cache size in megabytes, not bytes.
* kvutils: Move cache building from `Config` to the `caching` package.
* kvutils: Replace Guava's cache with Caffeine.
* kvutils: Simplify caching configuration.
* sandbox: Enable state value caching by default.
CHANGELOG_BEGIN
- [Sandbox] State values deserialization is now cached, with a fixed
cache size of 128MB.
CHANGELOG_END
* Changelog commit.
CHANGELOG_BEGIN
- [Ledger Integration Kit] The state value cache is now opt-in, with a
default of no cache at all.
CHANGELOG_END
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
* sandbox: Move the events page size configuration value into config.
* sandbox: Pass `config` directly into JdbcIndexerFactory.
* sandbox: Reorder `eventsPageSize` before `metrics` in parameters.
* sandbox: Move `seeding` into `ApiServerConfig`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Name all parameters of `JdbcLedgerDao.writeOwner`.
Co-Authored-By: stefano.baghino@digitalasset.com
* participant-state-metrics: Wrap metric names in a value type.
For safety, and for simplicity when building upon prefixes.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use MetricName within MetricsNaming.
* kvutils: Use `MetricName` in `Committer`.
* sandbox | kvutils: Extract common metric prefixes.
* sandbox: Remove a redundant visibility modifier in `MetricsNaming`.
* participant-state-metrics: `MetricName` doesn't need to be a case class.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* kvutils: Use `MetricRegistry.name` to create metric names.
String interpolation is 4TL.
* sandbox: Make the metric names of the services consistent with kvutils.
CHANGELOG_BEGIN
- [Sandbox] Move the service metrics from ``daml.sandbox.indexService``
and ``daml.sandbox.writeService`` to ``daml.services.index`` and
``daml.services.write`` respectively. This brings Sandbox in line with
the Ledger Integration Kit.
CHANGELOG_END
* sandbox: Use `MetricRegistry.name` to create metric names.
String interpolation is 4TL.
* kvutils/app: Publish JVM metrics.
* kvutils/app: Sort the metrics prefixes.
* kvutils/app: Allow for specifying a metrics reporter as a CLI argument.
No changelog as this is hidden.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Remove an unnecessary string interpolation mark.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-state{,-index}: Move Timed*Service classes from Sandbox.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Metrics for the various read, write, and index
services.
CHANGELOG_END
* kvutils/app: Add timing metrics for read/write/index services.
* participant-state: Move metrics-related code to another Bazel package.
* participant-state-metrics: Add to artifacts.yml.
* participant-state-metrics: Move TimedIndexService back into Sandbox.
Cuts down on dependencies like nobody's business.
* Sandbox: Reveal contract id seeding flag
CHANGELOG_BEGIN
- [Sandbox] Add support for random contract identifiers. See section
`Contract Identifiers Generation` section in
docs/source/tools/sandbox.rst
CHANGELOG_END
* kvutils/app|sandbox: Rename `system` to `actorSystem`.
* sandbox: Pass a materializer in to the StandaloneIndexerServer.
There's no need for it to construct one when the caller always has one
available.
CHANGELOG_BEGIN
CHANGELOG_END
* recovering-indexer: Use `SubmissionId` instead of `LedgerString`.
Where appropriate.
* sandbox: Use the materializer implicitly in JdbcIndexer.
* kvutils: Remove the unnecessary execution context from the test base.
* kvutils: Remove the unnecessary execution context from the writer.
* ledger-on-sql: Make a proper owner so it has a proper execution context.
This means the parallelization now needs to come from the test, so I've
augmented ParticipantStateIntegrationSpecBase to take a proper execution
context instead of the serial one that ScalaTest provides, with a
default of `ExecutionContext.global`.
* ledger-on-memory: Make a proper owner with a proper execution context.
* kvutils/app: Remove `executionContext` from LedgerFactory.
Shouldn't need it in `ResourceOwner`. I was bad.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-memory: Make ResourceOwners real classes.
* ledger-on-sql: Make the ResourceOwner a real class.
* ledger-on-sql: Cause side effects on resource acquisition.
Not on owner construction.
Currently sandbox only supports TLS if you also enable client
authentication. There is no reason for why this has to be the case and
for things like DABL we want TLS without client authentication so it’s
useful to be able to test this in sandbox. This PR introduces a
`--client-auth` flag that allows you to configure the behavior. The
default is the current one of requiring client authentication.
This PR does not yet update Java clients, however, the Haskell client
supports this already and is used to test this functionality.
I’ve also added a section in the documentation on TLS (there were no
docs at all so far).
changelog_begin
- [DAML Sandbox] When Sandbox is run with TLS enabled, you can now
configure the requirement for client authentication via
``--client-auth``. See
https://docs.daml.com/tools/sandbox.html#running-with-tls for more information.
changelog_end
* ledger-api-test-tool: Fix warnings flagged by IntelliJ IDEA.
* ledger-api-test-tool: Open-world mode.
In open-world mode, parties aren't allocated; their names are just
reserved for the test case, so that no other test will accidentally use
the same party name.
This is so we can test ledgers which dynamically allocate parties, such
as Sandbox.
* sandbox: Run conformance tests in "open-world" mode.
This means that the tests don't explicitly allocate parties (except for
a few), instead relying on Sandbox's implicit party allocation feature.
This is not enabled for Sandbox Next yet.
* sandbox-next: Implicit party allocation.
This is added to the command submission service.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-next: Don't implicitly allocate pre-existing parties.
* ledger-api-test-tool: Move pre-allocation into ParticipantTestContext.
* ledger-api-test-tool: We can reserve parties or wait for them. Not both.
Make illegal states unrepresentable as early as possible.
* sandbox: Name ApiSubmissionService's private methods a little better.
* sandbox: Move ApiSubmissionService's conditional logic into methods.
* sandbox: Document why we set `implicitPartyAllocation` to `false`.
* sandbox: Document why `implicitPartyAllocation` is dangerous.
* sandbox: Re-use the root actor system in the StandaloneIndexerServer.
* kvutils/app: Don't use the ActorSystem execution context randomly.
Instead, make `Runner` a proper ResourceOwner, with an `acquire` method.
* sandbox: Re-use the root actor system in the StandaloneApiServer.
CHANGELOG_BEGIN
CHANGELOG_END
* resources: Remove the now-unused `ResourceOwner.sequence` functions.
They weren't well-thought-out anyway; they acquire resources
sequentially, rather than in parallel.
* Allow `LedgerFactory` to provide a full-blown `ReadWriterService` rather than a `LedgerReaderWriter` (needed by at least vDAML)
CHANGELOG_BEGIN
CHANGELOG_END
* Address review points
* Address Samir's review point in `SqlLedgerFactory`
* Finish addressing Samir's review point in `SqlLedgerFactory`
* Split reader and writer owners in `LedgerFactory` as suggested by Gerolf
* Remove unneeded `val` from `KeyvalueParticipantState[Reader|Writer]` constructor params
* Remove unneeded type parameter from `app.Runner`
* Leave `LedgerFactory` a full ledger builder but split hierarchy upwards and clarify responsibilities
* Rename `SimpleLedgerFactory` to `KeyValueLedgerFactory`
* sandbox-next: Make the Runner a real ResourceOwner.
* sandbox: Don't construct the ResetService twice.
* sandbox: Inline and simplify methods in StandaloneApiServer.
* resources: Define a `ResettableResource`, which can be `reset()`.
`reset()` releases the resource, performs an optional reset operation,
and then re-acquires it, binding it to the same variable.
* resources: Pass the resource value into the reset operation.
* sandbox: Fix warnings in `TestCommands`.
* sandbox-next: Add the ResetService.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Make sure the SandboxResetService resets asynchronously.
It was being too clever and negating its own asynchronous behavior.
* sandbox-next: Forbid no seeding.
This double negative is really hard to phrase well.
* sandbox-next: Implement ResetService for a persistent ledger.
* sandbox: Delete the comment heading StandaloneIndexerServer.
It's no longer meaningful.
* sandbox-next: No need to wrap the SandboxResetService in an owner.
* sandbox-next: Bump the ResetService test timeouts.
It looks like it's definitely slower than on Sandbox Classic™. Gonna
look into this as part of future work.
* Revert to previous asynchronous reset behavior
Co-authored-by: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* libs-scala/ports: Wrap socket ports in a type, `Port`.
* sandbox: Use `Port` for the API server port, and propagate.
CHANGELOG_BEGIN
CHANGELOG_END
* extractor: Use `Port` for the server port.
* ports: Make Port a compile-time class only.
* ports: Allow port 0; it can be specified by a user.
* ports: Publish to Maven Central.
