* participant-integration-api: Split `LedgerConfigProvider` into two.
We now have a read-only `CurrentLedgerConfiguration`, and a write-only
`LedgerConfigProvisioner`.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Improve logging in LedgerConfigProvisioner.
Making use of the logging context.
* participant-integration-api: Extract a CurrentLedgerConfiguration trait.
This highlighted a few issues in the tests where we were unnecessarily
calling methods.
* participant-integration-api: Split LedgerConfigProviderSpec.
We can test the two components individually now.
* participant-integration-api: Add a test for streaming the config.
* participant-integration-api: Add tests for provisioning configuration.
* participant-integration-api: Move configuration code into a new package.
* participant-integration-api: Fake ledger config in tests.
Now that there's an interface, we can use it without worrying about
streaming from the index.
* participant-integration-api: Use `List`, not `Seq`, in test fixtures.
Scala 2.12 thinks `Seq.apply` yields a possibly-mutable sequence, which
is incompatible with `Source.apply`.
* participant-integration-api: Explain how `LedgerConfigProvider` works.
* Stop execution when conflicts are detected and remove any retries.
As the current logic for retries had a flaw where we would continue the flow even if we had conflicts, we're removing the retries and fixing the flaw by dropping the transaction. In the future we might add back the retries if we need them.
CHANGELOG_BEGIN
Fix bug where post-execution conflict detection was ignored in the pre-execution workflow
CHANGELOG_END
* Extract common pre-execution test code
* Add test for pre-execution conflict detection
* Finish submission aggregator only on actual success
* Code review formatting
* Use recover to handle conflicts, simpler and clearer than transform
* Update test to verify submission is not submitted when dropping transaction
* logging-entries: Make `LoggingEntries` a non-case class.
There's no reason for it to need `equals`, etc.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Convert commands into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This also adds a couple of missing items, and always logs `workflowId`.
* participant-state: Convert updates into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This changes some of the logging context structure, but otherwise
everything remains the same.
* Make sure Scaladoc is lined up for modified code.
- Drop the wrapper `Versioned` and use directly the `VersionedValue` proto class. Values
should not be handled alone outside the Engine.
- Add benchmarks to aggregate (de)serialization and (en/de)coding
- This also fixes a recursion limit decoding issue that appears with #10393.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-state: Give `ChangeId` its own file.
* participant-state: Introduce `v1.CompletionInfo` for a while.
`CompletionInfo` is identical to `SubmitterInfo`. Its purpose is to make
the migration to v2 easier. It should not make it into the final
version.
* participant-integration-api: Switch to the v2 participant state API.
This means that:
- the API server and indexer expect v2 API traits
- adapters are provided if you need to elevate your v1 API usage
- the indexer internally uses v2 Updates
- rejections are switched over to use the v2 format
- Sandbox Classic uses v2 as the internal representation too (because
it interacts directly with the underlying indexer representation,
and is therefore tightly coupled)
_kvutils_ and other users of the `StandaloneApiServer` and
`StandaloneIndexerServer` use the adapters.
CHANGELOG_BEGIN
- [Integration Kit] The API server and indexer have switched over to v2
of the participant-state API. You can continue to use the v1 API, but
you will need to wrap your ``ReadService`` and ``WriteService``
objects in the ``AdaptedV1ReadService`` and ``AdaptedV1WriteSerivce``
classes.
CHANGELOG_END
* participant-state: Remove v1.CompletionInfo.
It's served its purpose.
* kvutils: Remove an unnecessary line from `Runner`.
* ledger-api-common: Delete a TODO; we'll track it elsewhere.
* participant-integration-api: Use full words in `JdbcLedgerDao`.
Just a little bit of cleanup.
* ledger-api-common: Extract out the random submission ID generator.
And introduce a trait, because, well, this is still the JVM.
@remyhaemmerle-da is taking care of 1.16.0-snapshot.20210727.7476.0.b5e9d861 (#10428), so they get pushed back to the end of the line.
Please do not merge this before #10428.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@remyhaemmerle-da is in charge of this release.
Commit log:
```
b5e9d86114 Moved DamlSubmission* into separate proto file [KVL-980] (#10362)
0da814d250 Let stable packages bypass LF version restrictions. (#10377)
c97cbcabfd [JSON-API] Validate schema version & add minimal options for schema creation (#10374)
ebb8fab97e Add a ContractDao benchmark for payload queries (#10426)
85af078085 LF: parser for LF versions (#10424)
b976c9cf4c Disable autocommit for hikariCP for http-json-api db conn (#10427)
3ca46a4bcb Removed unused import. (#10425)
4d1249390e Introduce buf checks [KVL-980] (#10411)
1c4ae5023d Revert "Upgrade hikari to latest jdk8 version (#10406)" (#10421)
fe1b642e02 Don't fail if logs directory exists already (#10423)
7c88b56e16 participant-integration-api: Fix completion debug log output. (#10415)
fc305e6c2e [JSON-API] Shutdown on startup if the db connection is invalid (#10360)
6db58695ad Update vcredist (#10417)
4b55f1ab6a Connection pool for Contract Dao (#10359)
72cf2f36d3 LF: replace bazel keyword stable by default (#10410)
2bdcb7b69f update NOTICES file (#10414)
39c6e0b329 Fix oracle message too long error (#10413)
2094e24f28 Indexer ValidateAndWaitOnly startup mode for canton participant HA (#10290)
ad13a86947 Windows dev-env Powershell 7 compatibility (#10408)
3cedd83447 Easy to parse ledger-api-bench-tool logs (#10320)
16ff20c04f Fix links in changelog (#10409)
b325e8a8b8 participant-state: Remove `WriteService#rejectSubmission`. (#10407)
9be520c3e8 Keep the participant-state API prefixed. [KVL-1002] (#10405)
d88dc7129b Upgrade hikari to latest jdk8 version (#10406)
8b337bda4b Publish ledger-indexer-benchmark (#10401)
9e05f38f51 ledger-api-domain: Store the deduplication duration in `Commands`. [KVL-1002] (#10403)
9c064dac35 Allow imports of internal modules (#10397)
b9518ce2f4 participant-state-metrics: Add wrappers for v2. (#10404)
c3a3d60083 don't call Gary, he's on holiday (#10400)
221d0a0689 use doobie 0.9.0 Fragment-in-Fragment interpolation in json-api db-backend (#10399)
17709b5ba3 use a single SQL query for any number of json-api query pairs (#10344)
6a166841c5 Stop publishing the db-backend artifact (#10396)
1bed05fec2 Treat KeyWithMaintainers field structurally in all node types. (#10392)
d7077e154f Introduce locally-defined `Rejection` reasons instead of the participant-state rejection reason type. [KVL-1002] (#10376)
96f048330a [Divulgence pruning] Conformance tests implementation [DPP-484] (#10385)
28c5e9a435 update NOTICES file (#10386)
387945298d fix cut&paste typo bug; there are no insig lookup tweaks! (#10389)
7df9758c2f Daml export: make paths relative to daml.yaml (#10388)
90c3582ee8 treat exerciseResult structurally for isReplayedBy (#10381)
27a0c694bc Stop swallowing error message in non-repudiation conformance tests (#10387)
22b6101bc8 rotate release duty after 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10346)
5242e2c618 LF: drop old serializability check for Values (#10382)
ee755306fe LF: Specify nesting constraint for serialized values. (#10375)
4a33c03f72 LF: Add check of nesting in SValue.toValue (#10370)
91529ee43d clear up record specifications in http-json integration tests (#10366)
e8247afc6e update compat versions for 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10354)
74751ba9a8 Populate workflow-id in the test-tool scenarios (#10372)
da9f8e3007 clean-up perf tests (#10355)
42b70ad31d Fetch actingParties always non-empty for supported versions (>=1.6) (#10357)
37ff1a60fe ledger-configuration: Return a structured error from checkTime. [KVL-1002] (#10373)
66284c17f2 Limit length of package ids to 64 characters (#10368)
a56cfead28 even earlier mount failure detection (#10371)
1bc0ccdac7 update NOTICES file (#10367)
9c9b91eca7 Support deletion of a large number of contracts (#10353)
1b5f99e051 Stop printing stacktrace on logging setup failures (#10364)
c0a24fe9b6 HA PoC behind a feature flag [DPP-426] (#10227)
63739fa712 Add conformance test for deeply nested values (#10319)
faf479e8c8 LF: add context in LookupError (#10314)
68dcda0f29 Drop unused textType from JSON API Oracle queries (#10356)
49745f6e47 Re-add `application_id` to `DamlCommandDedupKey` [KVL-1000] (#10341)
27d439d546 LF: Compile netsed ELets using constant stack (#10337)
628aa22f51 kvutils: Refined transaction validation [KVL-1015] (#10066)
d1e84c8563 participant-integration-api: Fill out stubs in ApiSubmissionServiceSpec. (#10349)
905d8add44 remove duplicated rule in create-daml-app dlint (#10352)
bdc3e509b3 Separate ledger export related message from other kvutils messages [KVL-980] (#10343)
70e8ff46da participant-integration-api: De-spacify PostCommitValidationSpec. (#10350)
186d279c81 release 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10345)
e58c7ba3b4 fix template dot-files (#10342)
cf2b61e6ac participant-state: Remove the aliases to `Ref.LedgerString`. (#10325)
8148137259 [JSON-API] Include the logging context in the ledger client for consistent logging (#10332)
c9666c7fe0 Drop unnecessary mutability from speedy OnLedger state (#10340)
60dd96a3eb update NOTICES file (#10347)
```
Changelog:
```
[Integration Kit] Moved definitions of `DamlSubmission` and `DamlSubmissionBatch` to a separate proto file under the package `com.daml.ledger.participant.state.kvutils.wire`. In case you are directly referencing these messages you will have to update your imports.
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
- [JSON-API] The json api now correctly shutdowns at startup if the provided db connection is invalid in case of `createSchema=false`
simplify oracle migration scripts
- [Daml Compiler] Imports of internal modules from stable packages are
no longer illegal. Previously, the compiler raised an error when it
encountered imports of internal modules such as
`DA.Internal.Template`. Such imports are now accepted by the compiler.
Note, however, that internal modules are still not part of the stable
API. Fixes https://github.com/digital-asset/daml/issues/10379
[Integration kit] Extended the Ledger API test tool with tests for the pruning of all divulgence events.
- [Daml export] The generated paths to data-dependencies DALFs are now
relative to the generated daml.yaml. Fixes
https://github.com/digital-asset/daml/issues/10378.
- [JSON API] Fix an error where transactions that delete a large
number of contracts resulted in stackoverflows with the PostgreSQL
backend and database errors with Oracle.
[Integration Kit] The command de-duplication key now also includes the daml application ID
* [Integration Kit] Made `daml_kvutils.proto`'s location follow its proto package and moved `LedgerExportEntry` into a separate proto file. You may have to update your proto import statements in case you are directly importing proto files from the kvutils library.
- [JSON-API] Connection tries from the json api to the ledger now include the logging context, more specifically the instance_uuid is included in each logging statement.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
CHANGELOG_BEGIN
[Integration Kit] Moved definitions of `DamlSubmission` and `DamlSubmissionBatch` to a separate proto file under the package `com.daml.ledger.participant.state.kvutils.wire`. In case you are directly referencing these messages you will have to update your imports.
CHANGELOG_END
* Stable packages bypass version restrictions.
Part of #10029
changelog_begin
changelog_end
* Add missing bypass
* scalafmt
* Fix tests
* fix script runner
* Fix scenario service
* Dont make stable packages configurable
* unused import
* revert unnecessary formatting changes
* Added a StableOnly engine mode and a conformance test that uses it
* buildifier-fix
* Inline LanguageVersions.StableOnly
* scalafmt
* Remove EngineMode in favor of representing the version range directly
* Add -unsafe to flag name
* Apply suggestions from code review
* [JSON-API] Validate schema version & add minimal options for schema creation
* Add tests
* [JSON-API] Rework prior work and introduce the object SchemaHandling
* Add license headers & revert formatting changes
* Fix oracle build & scala 2_12 build
* correctly fix 2.12 build
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/SchemaHandlingResult.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* [JSON-API] Change case names & add backwards compat (but deprecate createSchema=true)
changelog_begin
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
changelog_end
* Add info about deprecated createSchema field
* Fix build & improve logging
* Give suggestions on what option to take, to fix an outdated or missing schema
* Renaming of schemaHandling to DbStartupMode, added more tests & correct exit codes depending on how the db startup went
* Align name with sandbox
* Improve tests
* Only add new sql code which strictly uses the interpolation to align with other pr's & minimally adjust statements
* Minimize diff
* Add backwards compat test
* Fix scala 2.12 build & oracle integration tests build
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/Config.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Adjust code according to review request & tests & add a failure test
* If the call to initialize fails also log the error which was thrown
* Fix formatting
* Add missing collections compat import in integration tests
* Fix last build errors (scala 2.12) & use Either instead of Option for getDbVersionState
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* [JSON-API] Shutdown on startup if the db connection is invalid
changelog_begin
- [JSON-API] The json api now correctly shutdowns at startup if the provided db connection is invalid in case of `createSchema=false`
changelog_end
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Main.scala
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Otherwise Bazel fails to run on Windows server 2019. The symptoms are
the same as with previous vcredist version issues. I.e. any Bazel
command, even `bazel version`, exits immediately and silently.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Switch ContractDao to use a HikariCP connection pool
CHANGELOG_BEGIN
CHANGELOG_END
* missed conn pool changes for PostgresTest and ContractDaoBenchmark
* shutdown db access await threadpool and fix formatting
* custom pool sizes for Prod and Integration similar to DbTriggerDao
* cleanup contract dao connection pool
* simplify Dao shutdown
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* remove redundant config setting
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* fix code formatting issue, NonUnitStatments warning
* Remove ServerCertificateValidationCallback
Causes issues as in https://github.com/PowerShell/PowerShell/issues/13597
* Get-ChildItem -Name
As of powershell 7 `Get-ChildItem` returns absolute paths which breaks
the later path concatenation `"$Directory\$file"`. This change instructs
`Get-ChildItem` to only return the file name, not the full path.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* participant-integration-api: Keep the state API prefixed.
So we can easily change the version.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-classic: Keep the state API prefixed.
Thought it might help with something, it didn’t but no reason to stick
to an older version.
Note that 5.x don’t work on jdk8 so this is the latest we can get.
changelog_begin
changelog_end
