changelog_begin
changelog_end
Status quo:
ledger-api-domain's domain object contained three classes related to parties:
1. `PartyDetails`, the oldest, which didn't contain the metadata field and was used both in client and indexer code,
2. `ParticipantPartyDetails`, added recently during the participant-local metadata extensions, similar to the `PartyDetails` above but additionally contains the metadata field,
3. `PartyRecord` which held participant-local metadata extensions for a party.
Changes:
1) `PartyDetails` gets renamed to `IndexerPartyDetails` and together with `PartyEntry` gets moved to participant-state-index Bazel package and is removed from client code,
2) `ParticipantPartyDetails` gets renamed to `PartyDetails` and is now used in the client code instead.
3) `PartyRecord` gets moved to participant-local-store Bazel package
* Allow the possibility to have a shared StringInterningView
changelog_begin
changelog_end
* Solve TODO in RecoveringIndexerIntegrationSpec
* Addressed Marton's review comments
* Added thread-safety mentionsin string interning interfaces
* Adds exponential ID page retrieval to aid low stream initialization
* Adds Working-Memory concept to calculate parameters to aid guaranteed memory footprint
* Switches to dynamic BatchN batching instead of static grouping to aid stream initialization
changelog_begin
changelog_end
Simplifies ID fetching logic
Removes dead code / tests
Cleans up factories (Materializer not needed anymore)
Simplifies ConcurrencyLimiter interface
Adds/cleans up unit tests
CHANGELOG_BEGIN
CHANGELOG_END
This PR fixes an issue with the ACS ID page retrieval: if there is one
of the id page DB fetching is slow, all the rest of the ID fetching
lanes getting fetched, due to continuous demand.
We implemented a treatment to cap the increased number of ID pages
accumulated in memory, by failing the stream.
Switching to mapAsync in pullWorkerSource naturally will
stop demand, in case a single query takes too long.
This also removes the need for the queue size limitting treatment.
* Switches to mapAsync in pullWorkerSource
* Fixes unit test
* Removes queue limiting feature (as it is not necessary anymore)
CHANGELOG_BEGIN
CHANGELOG_END
Continues the work started in https://github.com/digital-asset/daml/pull/12543
These libraries were only needed to transition from Scala 2.12 to 2.13
and are no longer useful as all the necessary items are now available
in Scala 2.13.
changelog_begin
changelog_end
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
This PR prepares the landscape for seamless integration of Index DB with non IndexService related
front components like UserManagementStore.
DbSupport is a simply a DbDispatcher and a StorageBackendFactory. Creation of it comes also in Flyway migrated flavor for capturing common need.
This is a pure refactoring PR: no behavior/feature changes
CHANGELOG_BEGIN
CHANGELOG_END
* [Self-service error codes] Enabled by default
* Flag changed to `use-pre-1.18-error-codes` (disabled by default)
CHANGELOG_BEGIN
[Ledger API Specification] The Ledger API returns enriched error codes (see https://docs.daml.com/error-codes/self-service/index.html)
For backwards-compatibility, a new API flag `--use-pre-1.18-error-codes` is introduced for preserving the legacy behavior for
clients that want to migrate incrementally to the changed gRPC status code responses and error details format.
CHANGELOG_END
* Adapted HttpServiceIntegrationTest
* Renamed `Feature Flag` to `Configuration` in docs
* Fix Daml Script tests
changelog_begin
changelog_end
* Fix Repl functests
changelog_begin
changelog_end
* Fix haskell binding tests
changelog_begin
changelog_end
* Fix CommandClientIT test
* Fixed Sandbox and CommandServiceBackpressureIT tests
Please enter the commit message for your changes. Lines starting
* Adapt //compiler/damlc/tests:repl-functests again
* Fix more tests and address Miklos' comments
* Flag name changed to `grpc-status-codes-compatibility-mode`
* Remove useless flags sandbox-classic
* Sandbox-classic tests fix for ContractKeysIT and ExceptionsIT
* Created 2 deprecated test suites that have the more generic assertions as returned
by the deprecated in-memory backend
* More fixes for CommandServiceIT
* Fixes compilation issue with the deprecated exceptionsIT class for Sandbox-classic in-memory
* Compatibility mode for old test tools
* Change flag name to `use-pre-1.18-error-codes`
* Apply suggestions from code review
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Wire to read pipelines (initialization and update)
* Patch StringInterning through the appropriate StorageBackend-s
* Creating a StringInterning for tests (which interns everything it sees)
changelog_begin
changelog_end
After this PR is merged write side of string-interning is activated:
* String interning table is being populated as part of usual indexing
* Data migration will populate the table from already existent data
Effect of populating string interning is not visible yet.
List of changes
* Add schema and data migrations where applicable
* Patch up sandbox-classic and parallel indexer to populate string_interning persistence, and initialize write side string interning view
* Add ledger_end_stringinterning_id to track the need for update at the view (+ aligning code)
* remove lookupLedgerEndOffsetAndSequentialId
* fix/extend unit-tests
changelog_begin
changelog_end
* Add a simple cache implementation
* Add support for ReadOnlySqlLedgerWithMutableCache
* Add support for ReadOnlySqlLedgerWithTranslationCache
* Add support for sandbox-classic
changelog_begin
changelog_end
* Remove the mutating schema
changelog_begin
- [Participant] All participants now use the new append-only schema. Existing databases will
automatically upgrade to the new schema the first time a participant/ledger is started.
changelog_end
* Fix post-commit validation test
* Remove append-only flags from CI
* Don't crash when using deprecated flag
* Increase timeout for reset service tests
* Fix typos in parameter names
* Restore removed test
* Restore removed CLI check
* Improve CLI parameter description
* Support contextual logging in KeyValueConsumption
CHANGELOG_BEGIN
CHANGELOG_END
* Fix formatting and imports
* Propagate the contextual logging per-readUpdates call
* No need for at-construction logging context in LogAppendingReadServiceFactory
* No need for logging context in CommitStrategySupport.newReadServiceFactory
* Fix JdbcIndexerSpec
* Propagate error version switch to KeyValueConsumption
CHANGELOG_BEGIN
CHANGELOG_END
* Introduce v2 (self-service) KV error codes behind the CLI switch
* Test v2 codes and fix them
* Keep newline separating methods in KeyValueConsumption
* TransactionRejections: don't wrap updates in `Some`
* Factor errorVersionsTable
* Reorder imports
* Split "convert rejection to proto models and back to expected grpc code"
* Remove unneeded Option returned by decodeTransactionRejectionEntry
* Formatting fix
* Fix 7537e93d
* kvutils: Remove `VersionedOffsetBuilder.apply`.
Doesn't do anything.
* ledger-on-memory: Use `VersionedOffsetBuilder`.
* indexer-benchmark: Use `VersionedOffsetBuilder`.
* ledger-on-sql: Use `VersionedOffsetBuilder`.
* kvutils: Use `VersionedOffsetBuilder` in tests.
* kvutils: Create a case class for `VersionedOffsetBuilder#split`.
* kvutils: Delete unused methods from the offset builders.
* kvutils: Use `Bytes#startWith` to check the offset version.
* kvutils: Move offset splitting into `VersionedOffset`.
* kvutils: Extract out versioned offset generators.
* kvutils: Replace `OffsetBuilder` with `VersionedOffsetMutator`.
This takes care of the last usages of `OffsetBuilder`, which were to
modify the lowest component of the offset.
CHANGELOG_BEGIN
CHANGELOG_END
* kvutils: Randomize the offset version in testing.
To make sure we don't use a hard-coded offset version anywhere.
* kvutils: `Random.between` is not available in Scala 2.12.
* kvutils: Move offset mutation methods to `VersionedOffset`.
* kvutils: Move the versioned offset construction into `VersionedOffset`.
The `VersionedOffsetBuilder` is still useful as it remembers the
version, meaning we only need to specify it once.
* Add deduplication and submission id to the proto submitter info so we can build the v2 completion info
* Use v2 rejections and store the enriched model in protobuf
* Update v1 SubmissionResult to v2 SubmissionResult
* Update v1 imports to v2 imports
* Enable API deduplication for ledgers that don't have it implemented yet (all kv)
* Update tests for new rejections
* Disable buf check for oneof
* Code review - fix formatting and test assertions
* Update imports for proto changes
* Add tests for rejection conversions
* Use entry id as submission id when submission id is missing
* Calculate deduplication period for rejection based on the set deduplicate until by calculating the duration relative to record time
* Code review consistent naming and comments
* Submission id is optional in the completion info
the submissionId is not set for participant.state.v1 entries, therefore for the transition from v1 to the v2 API we need to account for the missing submission ids by making it optional in the completion info.
* Update messages and grpc status codes for rejections to keep backward compatibility
* Deprecate proto deduplicate_until
* Set deduplication value in the commit context for all the deduplication periods. This change allows us to re-enable the committer deduplication tests
* Set ledger configuration for tests
* Command dedup: In the committer set the deduplicate until context value based on the given deduplication period [KVL-1049] (#10743)
CHANGELOG_BEGIN
participant-state:
- Migrate to use only v2 Read/Write Services. This includes the use of new models for rejections/updates/submission results.
- Calculate deduplicate until for committer side deduplication based on the submitter given deduplication period
- if using the deprecated deduplicateUntil then just set the given timestamp
- if using duration then calculate the new deduplicateUntil by using the formula (submissionTime + deduplicationDuration + minSkew)
- if using offset deduplication then calculate deduplicateUntil by using the formula (submissionTime + maxDeduplicationDuration + minSkew)
- if the deduplication period is not set then we don't set deduplicateUntil
- Emit completions with status `ALREADY_EXISTS` for duplicate commands
CHANGELOG_END
* Enable append-only schema in sandbox
changelog_begin
changelog_end
* Add test
* Unhide the compression flag in sandbox
* Make append-only config non-optional
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* participant-state: Give `ChangeId` its own file.
* participant-state: Introduce `v1.CompletionInfo` for a while.
`CompletionInfo` is identical to `SubmitterInfo`. Its purpose is to make
the migration to v2 easier. It should not make it into the final
version.
* participant-integration-api: Switch to the v2 participant state API.
This means that:
- the API server and indexer expect v2 API traits
- adapters are provided if you need to elevate your v1 API usage
- the indexer internally uses v2 Updates
- rejections are switched over to use the v2 format
- Sandbox Classic uses v2 as the internal representation too (because
it interacts directly with the underlying indexer representation,
and is therefore tightly coupled)
_kvutils_ and other users of the `StandaloneApiServer` and
`StandaloneIndexerServer` use the adapters.
CHANGELOG_BEGIN
- [Integration Kit] The API server and indexer have switched over to v2
of the participant-state API. You can continue to use the v1 API, but
you will need to wrap your ``ReadService`` and ``WriteService``
objects in the ``AdaptedV1ReadService`` and ``AdaptedV1WriteSerivce``
classes.
CHANGELOG_END
* participant-state: Remove v1.CompletionInfo.
It's served its purpose.
* kvutils: Remove an unnecessary line from `Runner`.
* ledger-api-common: Delete a TODO; we'll track it elsewhere.
* participant-integration-api: Use full words in `JdbcLedgerDao`.
Just a little bit of cleanup.
* ledger-api-common: Extract out the random submission ID generator.
And introduce a trait, because, well, this is still the JVM.
* participant-state: Remove the aliases to `Ref.LedgerString`.
Use the aliases directly in `Ref` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/data: Improve comments on ID aliases.
* daml-lf/data: Fix a typo in a comment.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-state: Remove the `ParticipantId` alias.
This alias adds nothing. By using `Ref.ParticipantId` directly, many
packages can remove their dependency on the _participant-state_ package.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Remove the `PackageId` and `Party` aliases.
They don't add anything. Let's just use `Ref`.
* kvutils: Restore missing compat imports.
* ledger-offset: Move `Offset` to a new package.
CHANGELOG_BEGIN
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
CHANGELOG_END
* Avoid rewrapping offsets for no reason.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* participant-integration-api: Sort some imports.
* participant-integration-api: Fix dependencies for the Oracle tests.
I didn't add `ledger-offset`.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* ledger-configuration: Extract configuration from participant-state.
The configuration is often used without the state, and doesn't need to
be versioned in the same way.
CHANGELOG_BEGIN
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
CHANGELOG_END
* participant-state: Remove the `LedgerId` aliases.
* ledger-configuration: Rename `TimeModel` to `LedgerTimeModel`.
This avoids confusion with the protobuf-generated `TimeModel` classes.
CHANGELOG_BEGIN
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
CHANGELOG_END
* ledger-configuration: Remove colons in LedgerInitialConditions' docs.
* kvutils: Restore a missing compat import.
* participant-integration-api: Add ledger-configuration to Oracle tests.
* sandbox-common: Fix `--max-ledger-time-skew` docs.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* logging-entries: Split from contextualized-logging.
This allows us to introduce it to Daml-LF without bringing in the
Logback, Logstash, and gRPC dependencies.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Fix dependencies for 2.12.
* logging-entries: Missed one more Scala 2.12 dependency.
* release: Publish logging-entries.
