* disable Any wart
* first pass removal of Any suppressions for false positives
* second pass removal of Any suppressions for false positives
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* third pass removal of Any suppressions for false positives
* fourth pass removal of Any suppressions for false positives
* reformat newly single-suppressions into single lines
- suggested by @SamirTalwar-DA; thanks
* Extract caching from participant-state as a library
This will be used to keep a cache of values to cut on LF translation cost when serving transactions.
changelog_begin
changelog_end
* Add dependency where missing
CHANGELOG_BEGIN
- [Sandbox] The ledger API server will now always use the most recent ledger configuration.
Until a ledger configuration is read from the ledger, command submissions will fail with the UNAVAILABLE error.
CHANGELOG_END
In kvutils, the first ledger configuration change needs
to have a generation one higher than the one returned
by getLedgerInitialConditions().
Remove initial config writing from sandbox as it's now written by the ledger API server
* Sandbox: expose back pressure config in CLI
CHANGELOG_BEGIN
[Sandbox]: Added ``--max-commands-in-flight`` as CLI configs. See ``daml sandbox --help``.
[Sandbox Classic]: Added ``--max-commands-in-flight`` and
``--max-parallel-submissions`` as CLI configs. See ``daml sandbox-classic --help``.
CHANGELOG_END
* Bumping the default maxParallelSubmissions to 512 for sandbox classic
We used to use `maxCommandsInFlight * 2` in SqlServer, but it makes more
sense to use `maxParallelSubmissions` there. Since the lower default value of 128
would result in the conformance tests to fail, I'm bumping it to 512
* Use maxCommandsInFlight to configure the parallel submissions for CommandService
* Add a reason text field to RejectReason.Inconsistent (#5180)
CHANGELOG_BEGIN
- Add a reason text field to RejectReason.Inconsistent.
See `#5810 <https://github.com/digital-asset/daml/issues/5810>`__.
CHANGELOG_END
* Change wording in contributing instructions to reflect best practice (#5820)
* Also make add reason text to other reject reasons that don't have it (#5820)
* Update with review comments (#5820)
* Update with review comments (#5820)
* Update with review comments (#5820)
- Only upload packages during the initial startup.
- Avoid loading packages during subsequent resets
- Share an engine between Ledger API Server and Committer
* Use a randomized H2 URL to simular in-memory
The reset service test assumes to get a completely new ledger for each
test case. But because we use H2 in-memory with db_close_delay=1 and the
same H2 database name, the second test case gets the remnants of the
first test case.
Since we know that sandbox in-memory uses an H2 in-memory URL, we can
simply use SandboxBackend.H2Database for ResetServiceInMemoryIT.
CHANGELOG_BEGIN
[Sandbox] Drastically lower the time needed to do a reset via the
ResetService.
CHANGELOG_END
In #5419 we set specific loggers to WARN or ERROR so that we avoid noisy
logging by default. However, if the user specifies the --log-level CLI
param, not only the root logger, but also other loggers set in logback.xml
should log at that level.
CHANGELOG_BEGIN
[Sandbox] Properly respect the --log-level CLI parameter
CHANGELOG_END
* Make the contract keys test fail if the transaction is not rejected by the committer
- First, this removes the assert statement in the DAML model, so that the
transaction validation doesn't fail with a transaction abort.
- Second, the gRPC error message should contain Disputed. `InvalidLookup`
should only be used in Sandbox Classic, but due to code sharing, the
validation that triggers this error is also run for Sadbonx.
* Fix the key lookup in kvutils
The current way of transaction validation and key lookup does not work
in the following scenario:
- the transaction coming from the participant has a negative key lookup
- the contract key state is loaded, which points to a contract that has
been created by another transaction since the command interpretation
on the participant
- the contract state for the contractId inside the contract key state
has not been loaded, therefore the visibility and activeness check in
`ProcessTransactionSubmission.lookupKey` returns `None`, even though
there is in fact a contract with such a key.
To mitigate this issue, the `lookupKey` function must not check
activeness or visibility. If the submitter is not allowed to load the
contract via a key lookup, the authorization check should catch that.
Any other situation that is inconsistent should result in a rejection of
the transaction.
A small side effect of this change is that the `lookupContract` function
might try to load a contract that wasn't loaded from the kv state
earlier. This only happens because we first rebuild the entire
transaction before we compare it to the original transaction.
Previously we threw an exception in this case, but this results in a rather
ugly abort of the reinterpretation of the transaction. Therefore the
`lookupContract` function just returns `None`. Eventually the
transaction will be rejected during validation of the nodes, because
the `LookupByKey` nodes will be different.
This would not happen if we reinterpreted the transaction lazily and
compared the nodes as soon as they are produced with the corresponding
node of the original transaction.
CHANGELOG_BEGIN
[kvutils] Fixed the validation of key lookups.
See issue `#5562 <https://github.com/digital-asset/daml/issues/5562>`__.
[Participant Server, Ledger API Test Tool] Use rejection reasons of the participant-state API instead of sandbox specific ones.
CHANGELOG_END
* Validate causal monotonicity of used contract keys
When using LookupByKey, the return contract isn't necessarily fetched or used
any other way. This means that we need an extra pass over all contract keys to
ensure that causal monotonicity is respected. To be able to do this,
there is now a new field `DamlContractKeyState.active_at`, which is set to the
same value `DamlTransactionEntry.ledger_effective_time`.
An additional check in `ProcessTransactionSubmission` uses this field to verify
causal monotonicity.
Instead, opt-in explicitly in _ledger-on-memory_ and _ledger-on-sql_.
Wrapping the operations can confuse other users of `LedgerStateAccess`.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-on-memory: Add metrics to the operations.
* ledger-on-memory: Speed up reads by using views.
* kvutils: Time event processing.
* ledger-on-(memory|sql): Make classes private and final where possible.
* kvutils: Factor out a TimedLedgerStateOperations class.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Report timing metrics for ledger state
operations.
CHANGELOG_END
* ledger-on-sql: Record log read metrics.
* ledger-on-sql: Record database transaction timing metrics.
CHANGELOG_BEGIN
- [Sandbox] Record ledger database timing metrics under "daml.ledger".
CHANGELOG_END
* ledger-on-sql: Time queries.
* metrics: Add graphs for read events and DAML-on-SQL.
* ledger-on-memory: Simplify the tuple swap in reading the log.
Co-Authored-By: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger/metrics: Move metric helpers to their own Bazel package.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use ledger/metrics.
* metrics: Rename `Metrics` to `Timed` and drop the `timed` prefix.
Importing methods is harder than importing objects.
* metrics: Publish to Maven Central.
Turns out not every participant can support seeding (yet).
CHANGELOG_BEGIN
- [Ledger API Server] Re-introduce an option to disable seeding. This
does not affect Sandbox.
CHANGELOG_END
* Add failing test
The test can produce false negatives,
but locally it fails 10 out of 10 times.
* Stop deduplicating commands after rejections
Fixes#5338.
CHANGELOG_BEGIN
CHANGELOG_END
* Only deduplicate successful transactions
* Use pass instead of pure unit
* Handle exceptions
* kvutils: Simplify calculating the weight of a Caffeine cache.
And remove an errant `println` that slipped through the cracks.
Thank you to @ben-manes for the tip!
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Make classes final and defs into vals.
Co-Authored-By: Stefano Baghino <stefano.baghino@digitalasset.com>
Co-authored-by: Stefano Baghino <stefano.baghino@digitalasset.com>
* kvutils: Cache state value conversions from bytes.
This seems to have a decent speedup in ledger-on-memory.
CHANGELOG_BEGIN
- [Ledger Integration Kit] Submissions now look up ledger values from a
cache where possible, improving performance when there's contention over
certain resources (e.g. common packages). The cache size currently
defaults to 64 MB.
CHANGELOG_END
* kvutils: Make the SubmissionValidator statue value cache configurable.
* kvutils: Report state value cache metrics.
* kvutils: Add a suffix to a Long literal because WartRemover is unhappy.
Strangely, it doesn't fail on my machine.
* kvutils: Extract caching out into its own file.
* kvutils: Move the `bytesToStateValue` call into `cache.get`.
* kvutils: Move caching to its own package.
* kvutils: Inject the state value cache.
* kvutils: Default to no state value cache.
* kvutils: Accept a state value cache size in megabytes, not bytes.
* kvutils: Move cache building from `Config` to the `caching` package.
* kvutils: Replace Guava's cache with Caffeine.
* kvutils: Simplify caching configuration.
* sandbox: Enable state value caching by default.
CHANGELOG_BEGIN
- [Sandbox] State values deserialization is now cached, with a fixed
cache size of 128MB.
CHANGELOG_END
* Changelog commit.
CHANGELOG_BEGIN
- [Ledger Integration Kit] The state value cache is now opt-in, with a
default of no cache at all.
CHANGELOG_END
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
* sandbox: Move the events page size configuration value into config.
* sandbox: Pass `config` directly into JdbcIndexerFactory.
* sandbox: Reorder `eventsPageSize` before `metrics` in parameters.
* sandbox: Move `seeding` into `ApiServerConfig`.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Name all parameters of `JdbcLedgerDao.writeOwner`.
Co-Authored-By: stefano.baghino@digitalasset.com
* Implement timed command deduplication in kvutils
This adds a field deduplication_time to DamlCommandDedupValue for
deduplication timeout checking.
* Bump kvutils version to 4
* Fix CommandTracker pulling commandResultIn multiple times
Now that the timeouts are generated out of band, we have 2
"unsynchronized" places that pull on commandResultIn.
Whenever we pull, we need to check that commandResultIn hasn't been
pulled before.
* Add inStaticTimeMode flag to enable command dedup in sandbox-next with static-time
Fixes#4624.
CHANGELOG_BEGIN
[kvutils] KVUtils now respects the command deduplciation time instead of
deduplicating commands forever.
CHANGELOG_END
* kvutils: Remove the LedgerEntry trait; it's no longer necessary.
This was introduced to allow for heartbeats, which no longer exist.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Make LedgerRecord a case class again.
We used to store the envelope as an array of bytes, which doesn't have a
value-based `equals` method and therefore should not be used in a case
class. We now use a `ByteString`, so this is no longer an issue.
* kvutils: Use `.view` in SubmissionValidator.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Don't compute missing inputs unless we're asked.
* ledger-on-memory: Do less in `InMemoryLedgerStateOperations`.
* ledger-on-memory: Use `RangeSource` instead of `OneAfterAnother`.
Should be faster to just take a slice.
* ledger-on-memory: Don't bother locking when reading.
We're only reading the log, which is append-only; we never mutate
existing data. This means we don't need to lock to read it.
* ledger-on-memory: Make it impossible to construct a state with data.
* participant-state-metrics: Wrap metric names in a value type.
For safety, and for simplicity when building upon prefixes.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox: Use MetricName within MetricsNaming.
* kvutils: Use `MetricName` in `Committer`.
* sandbox | kvutils: Extract common metric prefixes.
* sandbox: Remove a redundant visibility modifier in `MetricsNaming`.
* participant-state-metrics: `MetricName` doesn't need to be a case class.
Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Contributes to #4231.
Remove checkpoints from Participant Server storage:
- Removed code to store checkpoints in the index database.
- Remove existing checkpoint rows in ledger_entries and participant_command_completions.
- Removed ObservedTimeServiceBackend
This commit modifies the java migration V2_1__Rebuild_Acs. This is safe to do, because:
a) any even semi recent persistent sandbox had already gone through this migration and won't re-run it
b) A new database doesn't even have entries yet to migrate.
CHANGELOG_BEGIN
[DAML Ledger Integration Kit] Removed the ``Hearbeat`` state update.
[Sandbox] Checkpoints are no longer emitted in regular intervals in wall
clock time mode.
CHANGELOG_END
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
