This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@adriaanm-da is in charge of this release.
Commit log:
```
cfcdc13c6e Add deprecation output for sandbox classic (#11119)
ac5c52dbe4 DPP-609 Add StorageBackendTestsDeduplication (#11088)
55af7ad729 HttpServiceTestFixture provides a sandbox-classic ledger used for performance tests (#11128)
63ab3f3e50 Fix change ID references in services.rst (#11130)
31db15d555 Port error code changes (#11113)
9fd8182bbb move reusable functions from ContractsFetch to new fetch-contracts library (#11057)
c2084f6e14 Try and improve the explanation of compatibility test exclusions (#11114)
4df9b7c8f1 Rename command completion dedup time to duration in participant code [KVL-1057] (#10901)
11bc22d085 LF: builtins to create, signatory, and obersver on interface payload. (#11120)
d4cb1f945e KV ledgers: disable participant side command de-duplication [KVL-1083] (#11095)
b9a5a83175 Minor: fix trace logging in `TimeBasedWriteSetSelector` (#11117)
018e90854c [JSON-API] Make key_hash idx non unique (#11102)
429f43731a Remove HA indexer integration test on H2 (#11104)
9cffa1faf6 LF: check LF transaction protobuf aginst local snapshots (#11064)
d7ee278518 Optional table prefix for trigger service (#11047)
268b2d3da0 Command deduplication - test case that uses CommandService and CommandSubmissionService [KVL-1106] (#11098)
fac05f657b bypass value enrichment for the performance test (#11112)
4860271a53 Port file was not written to... where? (#11108)
a8d703b996 Increase akka server request-timeout for HttpServiceIntegrationTests (#11109)
37d94aaef6 Add unit for daily performance tests (#11107)
eb4ac8a397 Increase timeouts for resetting database (#11103)
db75f7da0f interface methods: Scala Typechecker (#11097)
d3e6f16c61 [JSON-API] Migrating tests to use sandbox next (#11034)
ffc8d6894f Data dependencies deserialize imports (#11037)
4b31bf088f update compat versions for 1.17.0 (#11075)
9fd6326ae7 interfaces: consuming/non-consuming iface choices (#11009)
229ce4776f Data dependencies serialize imports (#11038)
5d9ec65cc6 Add test case for parallel command deduplication using mixed clients [KVL-1090] (#11093)
d64d965230 kvutils: Print state updates before and after normalization. (#11096)
be216aa872 update NOTICES file (#11089)
517e866df5 Extract common code for command dedup conformance tests [KVL-1090] (#11092)
7a4963b43d add `--max-inbound-message-size` flag to `daml ledger export` (#11087)
26d10b267d Drop NodeId type parameter (#10921)
3cbd9229aa More NonEmpty functions (#10930)
bcd468625c interface methods: Speedy (#11076)
85adaab550 [DPP-417][DPP-595] Error codes switching - follow-up (#11074)
d7b280b60c Remove debug print in daml-lf/interpreter build (#11091)
1ed6428980 LF: move archive snapshots in a separate directory (#11081)
5e424f8d1c Command dedup conformance suites readme (#11051)
8290347f74 DPP-496 HA indexer integration test (#11033)
735c3090a3 Switch from InputStream to Byte Array at the binary content JDBC transport (#11072)
6ae3afa8fb fix perf reporting (#11073)
bf801a6e1b Fix SimpleDeduplicationBasic when participant deduplication is disabled (#11083)
62234dc763 [Short] Remove unused code (#11079)
c1d1521a14 interface methods: Scala AST (#11070)
7dd9c2d3f0 Remove expectations for internal failures from parallel command dedup tests (#11061)
51125991e5 Explicit discard in daml-lf/interpreter (#11067)
554b36c817 rotate release duty after 1.18.0-snapshot.20210928.7948.0.b4d00317 (#11059)
d156964f0c Release SDK 1.17.0 (#11062)
6d8cf7089c Handle interface methods in LF conversion (#11054)
87ecf1fe63 daml-ledger: better errors for non-json responses. (#11055)
ec2d26f31f [Divulgence pruning] Fixes divulgence pruning offset update query (#11046)
f13c6d61ec release 1.18.0-snapshot.20210928.7948.0.b4d00317 (#11058)
d1805a3000 More dependabot fun (#11063)
07b273aab5 update NOTICES file (#11060)
```
Changelog:
```
- [JSON-API] make key_hash indexes non-unique, this fixes a bug where a duplicate key conflict was raised on the query store when the same contract was being witnessed twice by two separate parties
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
- [Daml Assistant] Add support for `--max-inbound-message-size` flag to the Ledger Export tool.
Add cons, find, delete and deleteBy functions to NonEmpty module.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* split akka-streams and doobie utils from ContractsFetch to new fetch-contracts library
* move most stream components from ContractsFetch to new library
* fix packages
* make the fetchcontracts domain model work
* move transactionFilter to fetchcontracts
* lots of unused imports
* start incorporating fetch-contracts in http-json
* move toTerminates
* more unused imports; http-json compiles
* more fetch-contracts dep
* bring back HasTemplateId[ActiveContract]; integration tests compile
* whole ledger-service tree compiles
* fix oracle missing dep
* scoping some new library identifiers
* remove apiIdentifier aliases
* comment on Aliases
* remove toTerminates shim
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* unused bazel imports
* remove already-addressed TODO
- suggested by @akshayshirahatti-da; thanks
* Try and improve the explanation of compatibility test exclusions
CHANGELOG_BEGIN
CHANGELOG_END
* Fix scenario description
* Attempt #2 at trying and improving the explanation of compatibility test exclusions
* Rename Completion.deduplication_time to deduplication_duration
CHANGELOG_BEGIN
CHANGELOG_END
* Breaking protobuf change: regenerate `buf` image
Breaking-Proto: true
* Create a new grpc exception for each duplicate result [KVL-1099] (#10887)
* Create a new grpc exception for each duplicate result
The metadata in the exception is not thread safe, and when being converted into server headers netty.Utils.convertServerHeaders, it calls discardAll which mutates the metadata. Because this was reused for all duplicate exceptions then we got corrupted metadata.
CHANGELOG_BEGIN
CHANGELOG_END
* Do not call duplicate command exception twice
* Chore slow migration error removal (#10886)
* Avoid slow-progress timeout StoppedProgressing to avoid flakiness
CHANGELOG_BEGIN
CHANGELOG_END
* Make schema migration retries and backoff configurable
* Review feedback - use RetryStrategy.constant instead
* Remove unused tailrec import
* Simplifications and runF by Marton
* Rename config options from retry to attempt and default to 30 attempts
* Fix Parallel Indexer initialization issue [DPP-542] (#10889)
RCA: if at parallel indexer initialization some error happening, then a promise never completes, which causes an initialization future never complete
Expected: failure should be propagated, and recovering indexer should retry 10 seconds later
Actual: failure not propagated, a zombie future freezes initialization, preventing retries
Impact: this is a corner case - if no problems at indexer initialization, the issues is not surfacing
* Extracts critical logic into helper function initializeHandle
* Adds unit tests for initializeHandle
* Fixes issue by completing the promise in all cases
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-test-tool - Add conformance test for parallel command deduplication using CommandSubmissionService [KVL-1099] (#10869)
* Extract deduplication "features" into a configuration to be used around the tests.
Better naming for assertions that support sync and async deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Fix broken test and use consistency for tests
* ledger-api-test-tool - Add conformance test for parallel command deduplication
CHANGELOG_BEGIN
CHANGELOG_END
* Add import for 2.12 compat
* Add silencer plugin
* Split parallel command deduplication scenario into it's own test suite
* Add the parallel command deduplication test to append only ledgers
* Run parallel command deduplication tests for append only ledgers
* Apply suggestions from code review
Co-authored-by: fabiotudone-da <fabio.tudone@digitalasset.com>
* Code review renames
* Add compat import
* Run the test concurrently
* Further rename completions' `deduplication_time` to `*_duration` in participant's codebase
CHANGELOG_BEGIN
CHANGELOG_END
* Re-compute sha256s of migrations
* Fix expected error message
* Improve consistency of "deduplication time" Vs. "deduplication duration" references
* Revert Oracle schema whitespace change
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
Co-authored-by: nicu-da <nicu.reut@digitalasset.com>
Co-authored-by: Oliver Seeliger <46452754+oliverse-da@users.noreply.github.com>
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
* Extract common code for command dedup conformance tests
CHANGELOG_BEGIN
CHANGELOG_END
* Revert changes for parallel tests
* Fix compile warnings
* Disable participant deduplication for KV
CHANGELOG_BEGIN
CHANGELOG_END
* Always call submitRequestAndAssertDeduplication with parties
* Fix party in CommandDeduplicationBase
* Don't test KV-based command deduplication with static time as it's broken
* Don't disable participant-side command deduplication for sandbox-next on static time
* Revert "Don't test KV-based command deduplication with static time as it's broken"
This reverts commit 3dac41aa
* Add missing dependency
* Don't cross-test this ledger API test tool with existing versions
* Add the current PR to the list of PRs that resulted in exclusions
* Apply @garyverhaegen-da's suggestion for compatibility exclusions
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
* Add failing test that covers the bug
* Fix on conflict error for inserts into the contracts table
changelog_begin
- [JSON-API] make key_hash indexes non-unique, this fixes a bug where a duplicate key conflict was raised on the query store when the same contract was being witnessed twice by two separate parties
changelog_end
* move test to parent so as to test oracle query store
* make key_hash indexes non-unique
* use recordFromFields
Co-authored-by: Akshay <akshay.shirahatti@digitalasset.com>
* Fix typo postgres --> oracle
* Move tablePrefix into base jdbcConfig
* Add table.prefix in trigger service migrations
* Add tablePrefix to trigger service db table names
changelog_begin
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
changelog_end
* Add tablePrefix config test for trigger service
* Fix Oracle test
* Allow existing schema in trigger service
CHANGELOG_BEGIN
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
CHANGELOG_END
* Don't ignore CLI flag value
* Update triggers/service/src/main/scala/com/digitalasset/daml/lf/engine/trigger/dao/DbTriggerDao.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Use fragment interpolation
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Add test case for command deduplication which uses both the CommandService and the CommandSubmissionService
CHANGELOG_BEGIN
CHANGELOG_END
* Run mixed CommandService/CommandSubmissionService test for all the possible combinations of calls
* Port file was not written to... where?
Adds the information on where the port file could not be written.
From the current phrasing it appears the intention was to add the
information but ultimately wasn't.
changelog_begin
changelog_end
* Address https://github.com/digital-asset/daml/pull/11108#pullrequestreview-770115835
* interface methods: Scala Typechecker
Part of #11006.
I tried to match the behavior of the Haskell typechecker.
changelog_begin
changelog_end
* scalafmt
* Fix type
* Changes to use sandbox next for our integration tests
CHANGELOG_BEGIN
CHANGELOG_END
* remove sandbox classic dependency for HttpServiceTestFixture and perf tests
* rely on sandbox next fixture test class
* add missing dependencies for http-json-oracle
* changes based on code review comments
* Add tag to skip test case for scala_2_12 and also add a jdbc backend for sandbox spun up for perf tests
* Reduce size of contracts for archiving test
* Extend ModRef type to include ImpSpec
* Extract 'importOriginFromPackageRef' from 'genModule'
* emitModRef with EmptyImpSpec for all items in the metadata value 'imports'
* Generalize simpleImportTest into dataDependenciesTest, for tests with a multiple-module library and app
* Test that orphan instances are propagated in data-dependencies
changelog_begin
changelog_end
* interfaces: consuming/non-consuming iface choices
We add the consumption behaviour to the interface choice definition and
typecheck accordingly.
CHANGELOG_BEGIN
CHANGELOG_END
update to new ghc-lib, conversion implementation
* update ghc-lib
* pinning stackage on unix
* pin stackage on windows
* Extract 'convertQualifiedModuleName' from 'convertQualified'
* Generate stub definition with orphan module imports
* Test that generated $$imports value has the right contents
* Update InternedTypes test to expect $$imports-related types
changelog_begin
changelog_end
* Add test case for parallel command deduplication using mixed submission through the CommandService and the CommandSubmissionService
CHANGELOG_BEGIN
CHANGELOG_END
On integrity check failure.
This makes it easier to debug failures, as normalization can make the
state update mostly useless.
CHANGELOG_BEGIN
CHANGELOG_END
Add support for `--max-inbound-message-size` flag to the [Ledger Export](https://docs.daml.com/tools/export/index.html) tool.
CHANGELOG_BEGIN
- [Daml Assistant] Add support for `--max-inbound-message-size` flag to the Ledger Export tool.
CHANGELOG_END
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
* interface methods: Speedy
Part of #11006
changelog_begin
changelog_end
* scalafmt
* Remove InterfaceMethods test
* Update daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DecodeV1.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
* Add indexer stability test
changelog_begin
changelog_end
* Add test for DebugStorageBackend
* Reduce the log output for IndexerStabilitySpec
* Add Oracle test
* Rename file
* Increase timeout
* Rename DebugStorageBackend
* Address review comments
* Fixup rename
* fixup address review comments
* Refactor IndexerStabilitySpec
* Reset can only be called on an aborted read service
* Non-HA test can fail for multiple reasons
* Fix logback for Oracle tests
Otherwise logback complains about missing classes at startup
* Fix Oracle jdbc url
* Improve assert message
* Improve log output
Otherwise the user has to wait for 10min to get useful log output about the connection issue
* Change default log ids
The previous defaults are not valid for Oracle
* fmt
* interface methods: Scala AST
Part of #11006
This PR takes care of the Scala AST and decoder.
It leaves the encoder, type checker, and speedy for later.
changelog_begin
changelog_end
* scalafmt
* AstRewriter
@stefanobaghino-da is taking care of 1.18.0-snapshot.20210928.7948.0.b4d00317 (#11058), so they get pushed back to the end of the line.
Please do not merge this before #11058.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
