@garyverhaegen-da is in charge of this release.
This supersedes the failed #10971 and #10993.
Commit log:
```
ced4a27240 Include concurrency info in output (#10970)
a0b6800a93 Limit maximum number of concurrent tests (#10969)
a0fe0f6239 release 1.18.0-snapshot.20210921.7889.1.1b473c2b (#10993)
58c82b6e41 LF: reduce usage of NodeCreate.coinst (#10988)
f56ce2a757 kvutils: Add structured error data to the error metadata [KVL-1032] (#10964)
3f4dbf6f8e interfaces: Syntax highlighting for interfaces (#10987)
d52a2857da interface: Add test for conversion functions (#10991)
28b8d9a1f7 bump dotnet (#10979)
fe10ffbe8f Use ValueEnricher in ScenarioRunner. (#10897)
bdad7d5c99 Upgrade Java on Windows (#10965)
34390f7062 interfaces: Implemented conversions, added test. (#10982)
7740b60836 participant-integration-api: Improve the CommandConfiguration docs. (#10978)
04f322e2aa Bump resources for daml-ledger tests (#10984)
6d9922e1f1 Retry upsert of command deduplication on oracle and h2 [DPP-609] (#10976)
eb4f1b2a05 LF: Move lookup tests out of EngineTest (#10973)
99836d2610 Handle fetchByKey callback correctly in scenario runner (#10980)
23e6a2dddb Improve feedback from failing match in AuthMatchers (#10981)
e299103450 [Short] Small test tool's CLI parser refactor (#10923)
fc153a91b2 release 1.18.0-snapshot.20210921.7889.0.1b473c2b (#10971)
1b473c2b9a Switch trigger service time provider default to wall-clock time (#10967)
4acf34b5fc Add conversions from/to interfaces in Daml parser (#10954)
d50df11f77 interface: Add to/from interface in scala ast (#10960)
9f88e0919f Drop jdk8 override (#10963)
e0587bcdd2 Add a bit of detail to RegisteredTemplates documentation, motivated by: (#10962)
cebe6ed482 Refactor safetyStep in the simplifier. (#10948)
ac192fc6db interfaces: Add to/from interface in Haskell typechecker (#10951)
4b4d7a3af7 Remove update normalizer which was too aggresive (#10925)
66e1098070 Add tests for the `CompletionFromTransaction` converter [KVL-1104] (#10885)
724e50d739 interfaces: Add to/from interface in Haskell AST (#10945)
9a8d55ad13 Change slack link to discuss wiki. Fixes#10946 (#10947)
8b3b033fad LF: Test preprocessor resuming (#10936)
2edfc062a0 ifaces: name collision, typecheck fetch/exercise (#10896)
5dc15c6911 LF: rename language Interface to PackageInterface (#10938)
054c6abb63 Upgrade canton to a more recent version (#10944)
b8533d50e9 [JSON-API] Production/HA documentation. (#10903)
a33176265a Clarify `CommandTracker` [KVL-1104] (#10943)
5244643d16 Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
6cc42ee9f4 rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
88ef05e557 sandbox-classic: Only allow `--max-parallel-submissions` here. (#10941)
ac02dbdeb9 LF: Exhaustive test for valueTranslator. (#10927)
409c0b4f60 interfaces: Add to/from_interface in proto (#10937)
02c8a9dfb8 Split CommonStorageBackend (#10871)
dc71a6aea8 update NOTICES file (#10932)
0ba54a4bf9 Add a missing test case to the `CommandTrackerFlowTest` (#10939)
855ecdf1a7 [DPP-572] Add ledger API test case for verifying conformance to `--min-tls-version` flag. (#10898)
3e13e3d87e Switch to stable urls in scoop python manifest (#10933)
fc2c87d950 ledger-api-client: Generate a submission id if it's empty in the `CommandClient` [KVL-1104] (#10926)
04d8f753b2 Clean up sandbox-on-x conformance tests. (#10766)
b4541b5a4d Logging delay of submitted commands (#10912)
b8e21d8f48 Fix takeFilter for the test StreamConsumer (#10918)
906368d7e2 LF: exhaustive test for CommandPreprocessor (#10914)
61d214e451 Add fetch, exercise implementations for interfaces in speedy. (#10911)
d01f8e1c35 Fix flaky ApiConfigManagementServiceSpec test (#10922)
50291ed61b interfaces: scala typechecker implementation (#10867)
cac8391d4c Bump ghc-lib to include daml interfaces parser (#10747)
b6a6bf7aac [Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
308f938512 Dpp 494 unit testing ha coordinator (#10862)
30f74adc99 Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
9582e019ec LF: Refactor PreprocessorSpec test (#10909)
9b0fa29aec Separate exercise & fetch for interfaces from templates (#10908)
f4adee91ca Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
8a391189a6 Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
8e22bb6b2d Drop ContractId typeparameter from Value (#10827)
45437059d4 Drop cocreature as a codeowner from runtime things (#10906)
e6e81470bb Release 1.17.0 RC from second to last commit (#10904)
f08ac5f65c Desugar interface implements declarations (#10895)
b5648c0e3d Make `CommandTracker` distinguish submissions of the same command using `submissionId` [KVL-1104] (#10868)
```
Changelog:
```
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
```
CHANGELOG_BEGIN
CHANGELOG_END
The `main` branch build for #10971 failed. #10979 should have fixed it,
so let's try again.
This needs to be merged _after_ #10979.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.17.0-snapshot.20210921.7889.0.1b473c2b
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
Commit log:
```
1b473c2b9a Switch trigger service time provider default to wall-clock time (#10967)
4acf34b5fc Add conversions from/to interfaces in Daml parser (#10954)
d50df11f77 interface: Add to/from interface in scala ast (#10960)
9f88e0919f Drop jdk8 override (#10963)
e0587bcdd2 Add a bit of detail to RegisteredTemplates documentation, motivated by: (#10962)
cebe6ed482 Refactor safetyStep in the simplifier. (#10948)
ac192fc6db interfaces: Add to/from interface in Haskell typechecker (#10951)
4b4d7a3af7 Remove update normalizer which was too aggresive (#10925)
66e1098070 Add tests for the `CompletionFromTransaction` converter [KVL-1104] (#10885)
724e50d739 interfaces: Add to/from interface in Haskell AST (#10945)
9a8d55ad13 Change slack link to discuss wiki. Fixes#10946 (#10947)
8b3b033fad LF: Test preprocessor resuming (#10936)
2edfc062a0 ifaces: name collision, typecheck fetch/exercise (#10896)
5dc15c6911 LF: rename language Interface to PackageInterface (#10938)
054c6abb63 Upgrade canton to a more recent version (#10944)
b8533d50e9 [JSON-API] Production/HA documentation. (#10903)
a33176265a Clarify `CommandTracker` [KVL-1104] (#10943)
5244643d16 Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
6cc42ee9f4 rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
88ef05e557 sandbox-classic: Only allow `--max-parallel-submissions` here. (#10941)
ac02dbdeb9 LF: Exhaustive test for valueTranslator. (#10927)
409c0b4f60 interfaces: Add to/from_interface in proto (#10937)
02c8a9dfb8 Split CommonStorageBackend (#10871)
dc71a6aea8 update NOTICES file (#10932)
0ba54a4bf9 Add a missing test case to the `CommandTrackerFlowTest` (#10939)
855ecdf1a7 [DPP-572] Add ledger API test case for verifying conformance to `--min-tls-version` flag. (#10898)
3e13e3d87e Switch to stable urls in scoop python manifest (#10933)
fc2c87d950 ledger-api-client: Generate a submission id if it's empty in the `CommandClient` [KVL-1104] (#10926)
04d8f753b2 Clean up sandbox-on-x conformance tests. (#10766)
b4541b5a4d Logging delay of submitted commands (#10912)
b8e21d8f48 Fix takeFilter for the test StreamConsumer (#10918)
906368d7e2 LF: exhaustive test for CommandPreprocessor (#10914)
61d214e451 Add fetch, exercise implementations for interfaces in speedy. (#10911)
d01f8e1c35 Fix flaky ApiConfigManagementServiceSpec test (#10922)
50291ed61b interfaces: scala typechecker implementation (#10867)
cac8391d4c Bump ghc-lib to include daml interfaces parser (#10747)
b6a6bf7aac [Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
308f938512 Dpp 494 unit testing ha coordinator (#10862)
30f74adc99 Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
9582e019ec LF: Refactor PreprocessorSpec test (#10909)
9b0fa29aec Separate exercise & fetch for interfaces from templates (#10908)
f4adee91ca Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
8a391189a6 Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
8e22bb6b2d Drop ContractId typeparameter from Value (#10827)
45437059d4 Drop cocreature as a codeowner from runtime things (#10906)
e6e81470bb Release 1.17.0 RC from second to last commit (#10904)
f08ac5f65c Desugar interface implements declarations (#10895)
b5648c0e3d Make `CommandTracker` distinguish submissions of the same command using `submissionId` [KVL-1104] (#10868)
```
Changelog:
```
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.18 not 1.17
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Manual release process. @stefanobaghino-da is in charge of this release.
Commit log (since 1.17.0-snapshot.20210907.7759.0.35a853fd):
```
976ca400eb Force JSON API to refresh packages for GET requests to /v1/query (#10835)
e42cd3a1c6 LF: add interface to scala AST and decoder (#10830)
07b72d0f3e Update Ledger API's proto documentation [KVL-1094] (#10815)
648acf52f8 Fix typo in app-arch.rst (#10798)
dc9fbb5aa9 Stringify errors in create-daml-app before calling alert (#10831)
90b008ef0d Stop using `controller … can` syntax in ledger tests (#10814)
e490b69471 Clarifications of code ownership in /ledger and /ledger-api (#10742)
8cdeb7539a make auth middleware aware that actAs => readAs (#10825)
e263d43aca Ledger API test tool: test that record time is monotonically increasing [KVL-1053] (#10552)
e5793a317b Account for multi participant conformance tests when updating the config [KVL-1092] (#10788)
2e39d7b647 Deprecate `Commands.deduplication_time` field [KVL-1096] (#10802)
7824201b8d Get rid of extra new line and prune precision for all gatling values (#10820)
a03f52a15e Add ledger API tests about command deduplication information in completions [KVL-1057] (#10748)
61a07b1986 Add command line option max-deduplication-duration for sandbox and KV [KVL-1098] (#10816)
0800504955 Fix DuplicateSubmissionID test exclusions (#10819)
1ff34ffbe8 Remove workaround for akka supervisor issue (#10818)
6d347d4e88 Add daml interfaces to the Haskell LF AST (#10811)
cecb27fe16 Stop using controller can syntax in skeleton template (#10799)
8830eaa2ec Drop controller can syntax from compiler tests (#10807)
9038a8051b Update code documentation for `participant-state` API [KVL-1094] (#10812)
60ffb79fb1 Command deduplication - Participant side/committer side command deduplication conformance tests split [KVL-1093] (#10784)
1e6540f8f2 Stop using `controller _ can` syntax in daml-lf tests (#10805)
0db15e6759 Ledger API test tool: ensure that test identifiers are unique (#10804)
f1e378a60b rotate release duty after 1.17.0-snapshot.20210907.7759.0.35a853fd (#10809)
1464177bf8 release 1.17.0-snapshot.20210907.7759.0.35a853fd (#10808)
deb8a8b740 [Divulgence pruning] Daml SDK docs update [DPP-536] (#10764)
8405ad58fd Enhance InMemoryLedger to use the ValueEnricher (#10757)
```
Changelog (since 1.17.0-snapshot.20210907.7759.0.35a853fd):
```
- [JSON API] Fixed a bug that prevented the JSON API to be aware of
packages uploaded directly via the Ledger API.
- The auth middleware will now consider that actAs credentials are valid
for readAs requests.
- [Ledger API Specification] Introduced a new conformance test about recort time in completions being monotonically increasing
ledger-api-test-tool - CommandDeduplicationIT tests only participant side deduplication, it should be disabled for KV ledgers. Added KVCommandDeduplicationIT which should be run for KV ledgers
- [Ledger API Specification] `Commands.deduplication_time` field has been deprecated, please use `Commands.deduplication_duration` instead.
ledger-api-test-tool - CommandDeduplicationIT tests only participant side deduplication, it should be disabled for KV ledgers. Added KVCommandDeduplicationIT which should be run for KV ledgers
[Ledger API Specification] Participant pruning of all divulged contracts is fully implemented: Participant operators can choose to prune all immediately and retroactively divulged contracts, by setting the newly-added prune_all_divulged_contracts flag in the ParticipantPruningService/Prune request.
```
changelog_begin
changelog_end
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@akrmn is in charge of this release.
Commit log:
```
35a853fd30 Async logging for http-json-perf tests and publish logs as a pipeline artifact (#10803)
a349217f78 get oracle perf test output (#10800)
5f448f2603 Cleanup DataSource initialization (#10794)
0e250d02e3 Fix parallel ingestion initialization (#10797)
b8bd5e6399 interface PoC: protobuf definitions (#10796)
a1da025b78 FreePort draw from outside ephemeral port range (#10774)
945a86c3f2 Fix a race in a test of the RecoveringIndexer (#10792)
4093bbd58c fix macOS Bazel cache (#10795)
ed99fe5aa5 ParticipantPruningIT added events for canton safe-pruning and multi-node synchronization (#10775)
5b28aefbce Deprecate kvutils rejections generated only by participant state v1 [KVL-1090] (#10791)
116d6a5994 DPP-577 Use BIGINT instead of TIMESTAMP (#10761)
cdf4bf1138 Check compatibility of timestamps (#10793)
098a08c58b Add Moisés to release rotation (#10789)
7ae199f027 Follow-up to h2 storage backend user/password in url: Review feedback switch to regex and move to pure function (#10541)
a5999abf8e remove dead code (#10765)
4b7391b0d5 LedgerApiTestTool prints skipped (excluded) tests (#10726)
a0f0fb3684 [DPP-418] Enable outstanding test case for private key decryption (#10778)
d750666f8e Do not drop details when converting between gRPC `Status` classes [KVL-1084] (#10745)
5ec9a234db Fix link to the gRPC health checks in the driver for PostgreSQL docs (#10779)
8538fe97b4 HA protected ledger initialization for append only schema [DPP-551] (#10705)
23b0fe7ac2 Fix exclusions for command service tests (#10777)
c4e0a755d4 LF: drop ad-hoc ImmArray builders (#10763)
303ba90595 participant-state: Re-enable integration test for command deduplication [KVL-1089] (#10751)
793253ca87 bump cleanup threshold (#10771)
4d67684737 Fix flaky DBLock tests (#10770)
751587125f Fix exclusions for command dedup in compat tests (#10767)
50fecfb9ef Wrap missing label names in quotes (#10749)
b1c6e87803 fix claims check in auth middleware (#10768)
5595d55c79 [JSON-API] Use the token from incoming requests to update the package list (#10602)
dbafea0e48 Add unit tests for DBLockStorageBackend [DPP-495] (#10746)
e5a6d70182 Added buffer size metrics for getTransactions/getTransactionTrees (#10744)
f76c868ee4 update NOTICES file (#10762)
bb908d04fb Create a link to party management service in main Ledger API page. (#10138)
e45d852ed4 Fixes participant to do retries on startup as waiting for DB connectivity (#10759)
f68a12930b Report some oracle_json_perf numbers on slack (#10754)
7270ee3c71 Handle dynamic port in auth middleware client (trigger service) (#10755)
ff1308ee3d [Docs] Add info on logs on Kubernetes & metrics in the ops section (#10525)
d26739087f Update buf image [kvl-1049] (#10752)
d2180cf60b Update exclusion for command deduplication to include full version range (#10750)
41881ba2ba Command dedup: migrate kvutils to use v2 services [KVL-1049] (#10679)
4525b8c265 [JSON-API] vanilla oracle_perf ci job (#10688)
16df8a5e35 [Short] Remove unused code (#10719)
b28afcf7ae [DPP-438] Update docs on metrics that no longer use <party_name> in their name (#10728)
0662025f4c Clarify what the `buf` image is and how it should be used (#10741)
9071a05c76 sandbox-classic: Reintroduce SqlLedger tests for the mutable index. (#10722)
f576cdfd06 fix flaky test in RecoveringIndexer (#9619)
f6a75b42f3 ledger-api-common: Do not mock final classes. (#10733)
e9c8af5024 Bump ghc-lib to include dropped parsing code for generic templates (#10735)
862a2901fb rotate release duty after 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10731)
12cb92b431 update compat versions for 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10737)
73c0de8db4 [DPP-578] Temporarily disable flaky test - Mockito plugin issue (#10734)
963bcb17eb release 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10730)
c11323ddb1 LF: Refactor engine test reinterpret (#10724)
a995fa3cd2 [DPP-574] Update docs about TLS: encrypted private key (#10727)
66970b7226 Not sharing the absolute Deadline object. (#10713)
c6c304b778 Improve script error on invalid script identifier format (#10702)
b748fd6b67 Support TLS in Daml Helper --json-api requests (#10709)
```
Changelog:
```
kvutils - protobuf rejections generated by the participant state v1 API are deprecated (Inconsistent, Disputed, ResourcesExhausted, PartyNotKnownOnLedger)
[Docs] Fixed link to the gRPC health checks in the driver for PostgreSQL docs
participant-state: Emit completions (CommandRejected) for duplicate commands when using pre-execution
- Fix a bug in the auth middleware where insufficient credentials could
still give access to list of running triggers.
- [JSON API] The cli option `--access-token-file` is now deprecated. It
is not needed anymore and you can safely remove it. Reason is that
the operations which prior required a token at startup are now done
on demand using the token of the incoming request.
- [Docs] Information was added in the `Operating Daml` section on how to aggregate logs on Kubernetes in conjuction with Daml services & what options exists for exporting metrics from daml services (not Kubernetes specific)
participant-state:
- Migrate to use only v2 Read/Write Services. This includes the use of new models for rejections/updates/submission results.
- Calculate deduplicate until for committer side deduplication based on the submitter given deduplication period
- if using the deprecated deduplicateUntil then just set the given timestamp
- if using duration then calculate the new deduplicateUntil by using the formula (submissionTime + deduplicationDuration + minSkew)
- if using offset deduplication then calculate deduplicateUntil by using the formula (submissionTime + maxDeduplicationDuration + minSkew)
- if the deduplication period is not set then we don't set deduplicateUntil
- Emit completions with status `ALREADY_EXISTS` for duplicate commands
- [Java Bindings] DamlLedgerClient.Builder allows to set a timeout for command using `withTimeout`.
- [Daml Assistant] The `daml ledger` commands now accepts `--tls` in
combination with `--json-api` to access a JSON API behind a TLS
reverse proxy.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@aherrmann-da is in charge of this release.
Commit log:
```
f058c2f16f DPP-368 clean up flags (#10711)
90ad24fd6a [Divulgence pruning] Prune immediate divulgence [DPP-513] (#10691)
183934b89b Command dedup: add columns to completions in append-only schemas [KVL-1057] (#10652)
27c13334b6 LF: Drop outdated TODOs (#10725)
9be577a7fb Enable pruning in the sandbox-classic when the append-only schema is used [DPP-567] (#10708)
9f072aeba0 Ledger-API Conformance test for Contract ID V0 (#10717)
867547c579 DPP-368 enable append-only flag in sandbox (#10710)
bdc511eaa9 [DPP-438] Change open-ended metric names into static ones (by removing partyName part) (#10706)
0c820063f1 [Divulgence pruning] Prune all divulged contracts only after migration offset [DPP-483] (#10661)
2555dbb30b Use soft references for values in the caches (#10715)
7fd590694e Add LedgerClientWithoutLedgerId next to the LedgerClient (#10681)
856c69c425 participant-integration-api: Increase a test timeout. (#10721)
b86d07d498 remove non functioning oracle json array indices (#10720)
f5e17567f9 sandbox-classic: Remove default parameters in `SqlLedger.Owner`. (#10718)
9ef3377864 LF: Update specification with Contract ID Comparability check (#10703)
e5c4734780 enable JSON search index on Oracle by default (#10539)
1ded42f185 [DPP-418] Protect TLS keys - follow-up cleanup (#10696)
3fcd986f0b Introduce a new `Offset` format [KVL-1063] (#10668)
a5781a6682 update NOTICES file (#10714)
89855054b5 participant-integration-api: Use deadlines, not deduplication times, for expiring trackers. [KVL-1009] (#10704)
65025c26b8 sandbox-classic: Add ErrorInfo metadata for rejections. [KVL-1048] (#10707)
97bda3ca36 LF: V1 Contract ID check in Preprocessor (#10687)
c2f90efbba Add CLI option to force disabling of participant deduplication (#10698)
6016633bb7 Construct ParsedModule directly in Daml Repl (#10701)
bbdf16aacf DPP-368 unhide append-only CLI flags (#10697)
a41b1349df Use the tracker retention period as the maximum expiry time. [KVL-1009] (#10700)
e750ba5c45 Make warning less scary. (#10699)
5f120bdf87 rotate release duty after 1.17.0-snapshot.20210824.7647.0.640fb683 (#10660)
a17253fb63 DPP-535 Verify postgres version (#10577)
301ce53a71 participant-integration-api: Add tests for ApiCommandService. [KVL-1009] (#10689)
bd01a211f4 [DPP-418] Protect Participant TLS keys (#10629)
7ee13243e1 update NOTICES file (#10695)
7c392f32dc update NOTICES file (#10693)
3db654e9dd update NOTICES file (#10690)
eff09c010a ledger-api-client: Wrap command submissions in a new class. [KVL-1009] (#10683)
d54adb2543 Ledger-API: Conformance tests for contract IDs suffixing (#10654)
aa2e869ede [Divulgence pruning] Pass divulged contract arguments through kvutils Write/ReadService [DPP-535] (#10598)
1a7831390e Disable DeeplyNestedValueIT suite against canton in Daml repo (#10686)
b5f9be3fbc participant-integration-api: Standardize tracker retention naming. (#10682)
2aa632e3aa ledger-on-sql: Do not increment the dispatcher head on start. (#10684)
eabb19d7f7 [ledger-api] Add deduplication_duration to deduplication period [kvl-1047] (#10676)
96ad9b5ab8 [Divulgence pruning] All divulgence events pruning [DPP-483] (#10634)
0b7980d6eb Update rules_haskell (#10674)
284edfc4cd Fix FlywayMigrations datasource (#10666)
adbe65fae1 Document ActionFail vs CanAbort (#10657)
52e7a6d928 update compat versions for 1.17.0-snapshot.20210824.7647.0.640fb683 (#10667)
f42e6b6877 Expose pending contracts in triggers (#10672)
7cc698948c Add multiple ways of specifying deduplication [KVL-1047] (#10601)
53be19f86c participant-integration-api: Ensure that all waiting, failed, and closed trackers are cleaned up. (#10662)
b27cde6440 participant-integration-api: Move tracker code around, and tidy up tests. (#10663)
fc9d35935a Drop alternative rules from dlint config (#10646)
5204d3ce7a Include committers in PartialTransaction root context (#10665)
387c68b9eb Normalize transaction values within the engine (#10648)
430c1cccf3 release 1.17.0-snapshot.20210824.7647.0.640fb683 (#10659)
ef239fdd1a participant-integration-api: Move `TrackerMap` code around. [KVL-1009] (#10653)
```
Changelog:
```
- [Sandbox] - Participant pruning is enabled in the sandbox-classic when the append-only schema is used
- [Ledger Client Scala Bindings] A new variant of the LedgerClient class
was added called `LedgerClientWithoutLedgerId`. This class does not
need a ledger id at initialization. It was added to allow skipping
any checks at initialization for use cases where either the
ledger id is not known at initalization or no valid token can be fed
at initialization for checking the ledger id. Furthermore for each
classes `ActiveContractSetClient`, `CommandClient`, `PackageClient`,
`TransactionClient`, `VersionClient` now exists a variant which
doesn't depend on a ledger id at initialization and instead requires
one for every function as parameter. Moreover the existing classes
are extending these classes with overriding the methods and setting
the default of the parameter with the given ledger id from
initialization. The class `LedgerClientWithoutLedgerId` already
makes usage of these variants e.g. `PackageClientWithoutLedgerId`.
- [Ledger Client Scala Bindings] The function `transactionSource` of the
class `LedgerClientBinding` now optionally accepts a token which is
passed on to the unterlying call.
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
- [Integration Kit] Changes the Offset format to contain a version and therefore reduces the highest index size by one byte
- [Ledger API Server] The command deduplication time is no longer used
for determining the period of time to track the command before giving
up. Instead, the gRPC deadline is used. If no deadline is provided
(or if the deadline exceeds the command tracker retention period), the
tracker retention period is used instead.
- [Daml Repl] Fix a bug where bindings with out of scope types would result in error in following lines.
- [Sandbox, participant] Added a flag to enable a new append-only database schema.
This schema was designed to support significantly higher performance.
In a future release, all applications will automatically migrate to the new schema.
- [Ledger API Server] The command service now uses the tracker retention
period (typically specified with the ``--tracker-retention-period``
command-line argument) as the maximum time to wait for a command to
arrive on the completion stream. After this time, the command will
time out, though it may still complete in the future. Previously, the
deduplication period was used, but it was likely the tracker would be
terminated before that anyway.
The default tracker retention period is 5 minutes, unless otherwise
specified.
- [DPP-418] [Participant] Add support for supplying server's private key as an encrypted file and then decrypting it with the help of a secrets server.
[Integration Kit] KV-based ledgers pass contract instances through the Write/ReadService, removing the need
for backfilling divulged contract lookups.
Note: KV Ledgers that have been created before this change will still be relying on backfilling lookups of divulged contracts,
hence pruning of all divulged contracts may result in failing lookups for divulged contracts.
ledger-api - add `deduplication_duration` as a future replacement for `deduplication_time` in the command proto definition
ledger-api - Command deduplication period can now be specified by setting `deduplication_offset` instead of `deduplication_time` (only valid for v2 WriteService). This change is backwards compatible.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@SamirTalwar-DA is in charge of this release.
Commit log:
```
640fb68342 Make Index DB enable multiple party additions [DPP-546] (#10623)
b22de6893b LF: Contract ID suffix check in Preprocessor (#10642)
7b94b0674e Map shortened scala test suite names to long names on Windows (#10628)
6e4a24c28c participant-state: Generate correct gRPC error codes by v2 `WriteService` [KVL-1081] (#10656)
663781a159 Update curl 7.73.0 --> 7.78.0 (#10655)
a471f15aa8 Dpp-558 Fix startexclusive queries on oracle (#10649)
e99254fd3e Augment `completion.proto` with deduplication-related info [KVL-1057] (#10619)
a00608c8b3 participant-integration-api: Accommodate changes to max dedup time. (#10650)
29c546c949 [Divulgence pruning] Added `prune_all_divulged_contracts` to PruneRequest [DPP-534] (#10635)
dea57ca099 In-memory fan-out optimizations (#10558)
77eb366627 [JSON-API] key_hash field to speed up fetchByKey queries (#10631)
5001329c9d LF: Comparisons fail at runtime if comparing local vs global CIDs (#10630)
055be4b3cb Disable deprecation warnings for data-dependencies (#10647)
c1559357f6 clean-up gsg README (#10641)
8501832757 DPP-468 StorageBackend tests (#10529)
4e08b47dd1 update NOTICES file (#10645)
733590d591 ledger-api-health: Use the Scala health status values everywhere. (#10640)
5b837ec2d9 Ledger API: add `buf` checks [KVL-1045] (#10625)
f77cd0a81c participant-integration-api: Attempt to fix RecoveringIndexerSpec. (#10639)
9d7f60f03e participant-integration-api: Fix a flaky test. (#10637)
4a9331ca55 Upgrade Nixpkgs [KVL-1045] (#10624)
01b6e89551 update compat versions for 1.17.0-snapshot.20210817.7604.0.0c187853 (#10610)
b578b0e7cf Reminder to put an empty line between subject and body (#10638)
c0fbad1f6e participant-integration-api: Remove `limitMaxCommandsInFlight`. (#10626)
b4af6d1155 Canton testing: Mark one more DeeplyNestedValueIT test flaky (#10636)
e807f4a918 Upgrade to a newer canton version (post 0.27.0 snapshot version) (#10632)
c4513f2eea Oracle append-only schema: enable contract id index on participant_events_xxxx tables (#10633)
3598e0928c LF: Drop contract ID Freshness check (#10620)
37c999e237 ledger-on-sql: Increase the concurrency for conformance tests. (#10622)
46e8c7d762 DPP-460 Extract constant for event sequential IDs (#10564)
121047e0ea DPP-460 Parameter storage consolidation (#10472)
569612ab73 Drop broken symlink config (#10616)
6d0109fca8 Support $ in daml-lf identifiers in the parser (#10609)
c38703ec84 participant-integration-api: Store a status gRPC protobuf. [KVL-1005] (#10600)
0af5b49484 make FinalReason a case class (#10614)
8dd136fc7d bazel-tools: Replace `runner` with either `runner_with_port_check` or `runner_with_port_file`. (#10615)
19c3d28ebf Remove GenMissingString class because it is not used (#10608)
3227e860e0 Use the port file and dynamic port generation in client/server tests. (#10604)
fb19bcb7f3 fix gsg-trigger template (#10611)
b2077026d6 rotate release duty after 1.17.0-snapshot.20210817.7604.0.0c187853 (#10606)
975a5fb32b Move DeduplicationPeriod to ledger-api-domain [KVL-1047] (#10590)
f8a1820cc8 release 1.17.0-snapshot.20210817.7604.0.0c187853 (#10605)
64abf8aaa9 update NOTICES file (#10607)
```
Changelog:
```
[Integration Kit] Corrected gRPC error codes returned by v2 `WriteService` adaptor.
- [Ledger API Server] The API server manages a single command tracker
per (application ID × submitters) pair. This tracker would read the
current ledger configuration's maximum deduplication time on creation,
but never updated it, leading to trackers that might inadvertently
reject a submission when it should have been accepted. The tracker now
reads the latest ledger configuration.
- Update schema version for http-json-api query store with new key_hash field
- Improved performance for fetchByKey query which now uses key_hash field
participant-state - move `DeduplicationPeriod` to ledger-api-domain
dc4629fff9 update NOTICES file (#10580)
8b0a0e7bbc update NOTICES file (#10578)
4b8b67a1b5 Upgrade Scalatest to v3.2.9. (#10576)
41e60f7302 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898cc9 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90dffd Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc6d6 rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44fbd update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa7c0 Bump bazel max jvm memory (#10569)
1cc136c033 update NOTICES file (#10565)
c69880ccf1 ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f80c Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f36b6 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
Commit log:
```
0c1878530b test: test for platform independent dars (#10535)
3822a8c886 ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de564f Introduce dependency on Oracle image version (#10597)
f8c0a35940 rewrite trigger docs to follow gsg (#10509)
aec601f929 Fix badly versioned empty tx (#10596)
d92440471c daml ledger export: export all parties (#10588)
fb09b72f96 Improve divulgence warning message. (#10595)
fd9d8724fc Upgrade to Oracle 19.12 (#10589)
386965c466 Upgrade Flyway to v7. (#10594)
654d2eee1b Bump url-parse to address dependabot alerts (#10593)
01b8e3011f Simplify loading of logback file (#10592)
19bfdbee00 update NOTICES file (#10591)
5f7a3691a5 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f32d4 Upgrade path-parse to 1.0.7 (#10587)
5c8025269d ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccfc4e Normalize transactions & values as a separate pass (#10524)
99e1d78bb8 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc39cb Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2b96 Skip subject in changelog check (#10584)
244262dfac Handle imports for builtin types correctly in Daml export (#10575)
1fc281f07e ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15465 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb933 JSON API: log ledger connection errors at every attempt (#10581)
933b58e3aa Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef894447f Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52cdc changelog_begin (#10579)
dc4629fff9 update NOTICES file (#10580)
8b0a0e7bbc update NOTICES file (#10578)
4b8b67a1b5 Upgrade Scalatest to v3.2.9. (#10576)
41e60f7302 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898cc9 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90dffd Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc6d6 rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44fbd update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa7c0 Bump bazel max jvm memory (#10569)
1cc136c033 update NOTICES file (#10565)
c69880ccf1 ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f80c Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f36b6 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@remyhaemmerle-da is in charge of this release.
Commit log:
```
b5e9d86114 Moved DamlSubmission* into separate proto file [KVL-980] (#10362)
0da814d250 Let stable packages bypass LF version restrictions. (#10377)
c97cbcabfd [JSON-API] Validate schema version & add minimal options for schema creation (#10374)
ebb8fab97e Add a ContractDao benchmark for payload queries (#10426)
85af078085 LF: parser for LF versions (#10424)
b976c9cf4c Disable autocommit for hikariCP for http-json-api db conn (#10427)
3ca46a4bcb Removed unused import. (#10425)
4d1249390e Introduce buf checks [KVL-980] (#10411)
1c4ae5023d Revert "Upgrade hikari to latest jdk8 version (#10406)" (#10421)
fe1b642e02 Don't fail if logs directory exists already (#10423)
7c88b56e16 participant-integration-api: Fix completion debug log output. (#10415)
fc305e6c2e [JSON-API] Shutdown on startup if the db connection is invalid (#10360)
6db58695ad Update vcredist (#10417)
4b55f1ab6a Connection pool for Contract Dao (#10359)
72cf2f36d3 LF: replace bazel keyword stable by default (#10410)
2bdcb7b69f update NOTICES file (#10414)
39c6e0b329 Fix oracle message too long error (#10413)
2094e24f28 Indexer ValidateAndWaitOnly startup mode for canton participant HA (#10290)
ad13a86947 Windows dev-env Powershell 7 compatibility (#10408)
3cedd83447 Easy to parse ledger-api-bench-tool logs (#10320)
16ff20c04f Fix links in changelog (#10409)
b325e8a8b8 participant-state: Remove `WriteService#rejectSubmission`. (#10407)
9be520c3e8 Keep the participant-state API prefixed. [KVL-1002] (#10405)
d88dc7129b Upgrade hikari to latest jdk8 version (#10406)
8b337bda4b Publish ledger-indexer-benchmark (#10401)
9e05f38f51 ledger-api-domain: Store the deduplication duration in `Commands`. [KVL-1002] (#10403)
9c064dac35 Allow imports of internal modules (#10397)
b9518ce2f4 participant-state-metrics: Add wrappers for v2. (#10404)
c3a3d60083 don't call Gary, he's on holiday (#10400)
221d0a0689 use doobie 0.9.0 Fragment-in-Fragment interpolation in json-api db-backend (#10399)
17709b5ba3 use a single SQL query for any number of json-api query pairs (#10344)
6a166841c5 Stop publishing the db-backend artifact (#10396)
1bed05fec2 Treat KeyWithMaintainers field structurally in all node types. (#10392)
d7077e154f Introduce locally-defined `Rejection` reasons instead of the participant-state rejection reason type. [KVL-1002] (#10376)
96f048330a [Divulgence pruning] Conformance tests implementation [DPP-484] (#10385)
28c5e9a435 update NOTICES file (#10386)
387945298d fix cut&paste typo bug; there are no insig lookup tweaks! (#10389)
7df9758c2f Daml export: make paths relative to daml.yaml (#10388)
90c3582ee8 treat exerciseResult structurally for isReplayedBy (#10381)
27a0c694bc Stop swallowing error message in non-repudiation conformance tests (#10387)
22b6101bc8 rotate release duty after 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10346)
5242e2c618 LF: drop old serializability check for Values (#10382)
ee755306fe LF: Specify nesting constraint for serialized values. (#10375)
4a33c03f72 LF: Add check of nesting in SValue.toValue (#10370)
91529ee43d clear up record specifications in http-json integration tests (#10366)
e8247afc6e update compat versions for 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10354)
74751ba9a8 Populate workflow-id in the test-tool scenarios (#10372)
da9f8e3007 clean-up perf tests (#10355)
42b70ad31d Fetch actingParties always non-empty for supported versions (>=1.6) (#10357)
37ff1a60fe ledger-configuration: Return a structured error from checkTime. [KVL-1002] (#10373)
66284c17f2 Limit length of package ids to 64 characters (#10368)
a56cfead28 even earlier mount failure detection (#10371)
1bc0ccdac7 update NOTICES file (#10367)
9c9b91eca7 Support deletion of a large number of contracts (#10353)
1b5f99e051 Stop printing stacktrace on logging setup failures (#10364)
c0a24fe9b6 HA PoC behind a feature flag [DPP-426] (#10227)
63739fa712 Add conformance test for deeply nested values (#10319)
faf479e8c8 LF: add context in LookupError (#10314)
68dcda0f29 Drop unused textType from JSON API Oracle queries (#10356)
49745f6e47 Re-add `application_id` to `DamlCommandDedupKey` [KVL-1000] (#10341)
27d439d546 LF: Compile netsed ELets using constant stack (#10337)
628aa22f51 kvutils: Refined transaction validation [KVL-1015] (#10066)
d1e84c8563 participant-integration-api: Fill out stubs in ApiSubmissionServiceSpec. (#10349)
905d8add44 remove duplicated rule in create-daml-app dlint (#10352)
bdc3e509b3 Separate ledger export related message from other kvutils messages [KVL-980] (#10343)
70e8ff46da participant-integration-api: De-spacify PostCommitValidationSpec. (#10350)
186d279c81 release 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10345)
e58c7ba3b4 fix template dot-files (#10342)
cf2b61e6ac participant-state: Remove the aliases to `Ref.LedgerString`. (#10325)
8148137259 [JSON-API] Include the logging context in the ledger client for consistent logging (#10332)
c9666c7fe0 Drop unnecessary mutability from speedy OnLedger state (#10340)
60dd96a3eb update NOTICES file (#10347)
```
Changelog:
```
[Integration Kit] Moved definitions of `DamlSubmission` and `DamlSubmissionBatch` to a separate proto file under the package `com.daml.ledger.participant.state.kvutils.wire`. In case you are directly referencing these messages you will have to update your imports.
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
- [JSON-API] The json api now correctly shutdowns at startup if the provided db connection is invalid in case of `createSchema=false`
simplify oracle migration scripts
- [Daml Compiler] Imports of internal modules from stable packages are
no longer illegal. Previously, the compiler raised an error when it
encountered imports of internal modules such as
`DA.Internal.Template`. Such imports are now accepted by the compiler.
Note, however, that internal modules are still not part of the stable
API. Fixes https://github.com/digital-asset/daml/issues/10379
[Integration kit] Extended the Ledger API test tool with tests for the pruning of all divulgence events.
- [Daml export] The generated paths to data-dependencies DALFs are now
relative to the generated daml.yaml. Fixes
https://github.com/digital-asset/daml/issues/10378.
- [JSON API] Fix an error where transactions that delete a large
number of contracts resulted in stackoverflows with the PostgreSQL
backend and database errors with Oracle.
[Integration Kit] The command de-duplication key now also includes the daml application ID
* [Integration Kit] Made `daml_kvutils.proto`'s location follow its proto package and moved `LedgerExportEntry` into a separate proto file. You may have to update your proto import statements in case you are directly importing proto files from the kvutils library.
- [JSON-API] Connection tries from the json api to the ledger now include the logging context, more specifically the instance_uuid is included in each logging statement.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@robin-da is in charge of this release.
Commit log:
```
b7cf42d173 Upgrade doobie to version 0.13.4 (#10326)
1a723149d4 fix broken triggers link (#10336)
271f9daedd Added back DeduplicationPeriod.DeduplicationOffset [KVL-998] (#10324)
b701caab7d Report divulgence warning at commit location. (#10329)
742bdcbeea Describe prometheus in daml-on-sql README (#10307)
93d0ed6821 [Mutable cache] Resolve with full lookup on negative cache read-through lookups [DPP-501] (#10262)
f59951b755 document min npm version (#10330)
d982e16e99 fix besu listing (#10328)
c3bc26fd1c daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_. [KVL-1002] (#10323)
cf2f79db4e Register divulgence warning as a diagnostic (#10289)
8360e9f75a Mark Java Bot tests flaky (#10322)
a6bf892f72 [In-memory fan-out] CLI parameters update (#10250)
5fa49c3312 Disables flaky test suite of InstrumentedSource.bufferedSource (#10321)
504c231732 Use a combined template id, stakeholder index (#10315)
9a01065f29 participant-state: Remove the Daml-LF transaction and value aliases. [KVL-1002] (#10317)
159728d716 LF: use Either by default in all archive reader API (#10295)
449a72a86f increase ES memory (#10318)
80b65f6def [DOCS] Add documentation for the JSON API metrics (#10312)
6abb903a7f [JSON-API] Put the request & response bodies into the log ctx if the log level is debug (for specific statements) (#10298)
3c0010b38a detect mount issue earlier (#10313)
0e09f3ba08 Remove optLocation field from transaction nodes. (#10301)
e62852fbfd Make ledger API test tool's 'max-connection-attempts' into an option (#10311)
bd12170af2 participant-state: Remove the `ParticipantId`, `PackageId`, and `Party` aliases. [KVL-1002] (#10308)
fed44970ae Extend JSON API ContractDao query bench’s with different tpids (#10309)
bd35f80efa ledger-configuration: Correct the protobuf package. [KVL-1002] (#10302)
d69328d281 Ledger API Test Tool: add a `--max-connection-attempts` command line option [KVL-977] (#10297)
a3b861eae8 refresh es cluster (#10300)
63332d0667 update NOTICES file (#10306)
1ee53c0736 participant-state-index: Use `Ref` directly. (#10299)
0eba812109 Remove trace_context field from Ledger API [KVL-1021] (#10256)
88886beb8d ledger-configuration: Add tests for the `Configuration` decoding methods. [KVL-1002] (#10293)
cb29f34d4b ledger-offset: Move `Offset` to a new package. [KVL-1002] (#10296)
b899ae1d37 Codeowners: Add @nicu-da to the kvutils code review. (#10292)
f033bc2bbb LF: Clean legacy from archive proto + TEXT_TO_CODE_POINTS typo (#10286)
70b12f04d0 ledger-on-sql: Remove conformance tests for H2 and SQLite files. (#10291)
a9a0b70b85 ledger-configuration: Extract the configuration from participant-state. [KVL-1002] (#10279)
d96b54eb23 Use Text not String in exception docs (#10287)
53822537e4 Refresh json API materialized view on statement (#10285)
6e447c567e Patch export filtering of GHC.Types.[] in damldocs (#10282)
d565cae913 DPP-457 Add UpdateToDbdto tests (#10116)
814442eb80 update NOTICES file (#10284)
0493480e44 rotate release duty after 1.15.0-snapshot.20210713.7343.0.1f35db17 (#10268)
84094e0b6b update compat versions for 1.15.0 (#10281)
ba8e829c8d [In-memory fan-out] Oracle support (#10263)
a8f190214c LF: change type from Try to Either in archive module (#10277)
c84b37a67f Release SDK 1.15 (#10269)
980d96b44a Fix status check in collect_build_data (#10278)
191d3b82ce Add unit test to SequentialWriteDao [DPP-456] (#10265)
2271f93b69 Fix crash on duplicate submissionId (#10242)
906184ce1a LF: Simplify UniversalDarReader (#10271)
42c487ba96 [JSON-API] Refactor Endpoints.scala to use path directives etc. (#10274)
684040159f release 1.15.0-snapshot.20210713.7343.0.1f35db17 (#10267)
0df70fa56a Document bazel-testlogs directory in BAZEL.md (#10276)
44204bb932 Add a benchmark for contract insertion in the JSON API (#10272)
6107f8aa64 Ignore failure to upload log failures (#10270)
caf85a2270 LF: rationalize archive Parser/Reader/Decoder (#10239)
0043b81f00 Add a ContractDao benchmark (#10259)
21a9e62f27 Add disclaimer to participant state v2 API [KVL-998] (#10254)
6c63f96ba9 participant-integration-api: Move `SeedService` here. (#10261)
58c0b4694a kvutils: Simplify usage of Rejections [KVL-1015] (#10258)
41aba1ca39 kvutils: Move unrelated test cases to a proper test class [KVL-1015] (#10257)
```
Changelog:
```
- [JSON-API] You can now find a section `Metrics` in the http-json api documentation explaining how to enable metrics and which are available
- [JSON-API] Logging of the request and response bodies are now available for appropriate requests if the chosen log level is equal or lower than DEBUG. These can then be found in the logging context of the request begin & end log messages (The field names in the ctx are "request_body" and "response_body").
- [Integration Kit] The *ledger_configuration.proto* Protobuf definition
has been repackaged under ``com.daml.ledger.configuration``, and the
Java and C# packages have been renamed accordingly. If you are using
this Protobuf definition, you will need to update your imports. The
Maven artifact name has not changed.
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
Fix crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participants.
Extend ledger-api-test-tool to cover the duplicate submissionId cases.
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* release 1.15.0-snapshot.20210713.7343.0.1f35db17
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@akshayshirahatti-da is in charge of this release.
Commit log:
```
1f35db17c7 ledger-on-sql: Use random log entry ID allocation so we don't depend on `SeedService`. [KVL-1002] (#10255)
b8e2198873 Separate traces from warnings in engine. (#10253)
0a7f2b1a1b [JSON-API] Small refactoring to start using akka http routes internally (#10252)
6e8ec1d618 LF: Drop old depreated code (#10251)
de7a08fa7b [In-memory fan-out] Handle `null` submitters in getTransactionLogUpdates (#10248)
584169a2cc Participant state v1-to-v2 adaptors [KVL-1002] (#10246)
e4d7cd764a LF: Deprecate com.daml.lf.data.TryOps.Bracket (#10249)
b59f36591d [JSON-API] Correctly extract the request source URL/IP (#10244)
edaf541063 Adapt `ledger-api-bench-tool` to work with LR [DPP-424] (#10171)
6fe6ae075d LF: Make DarReader ZipEntries immulatble (#10243)
05e5218d06 Consolidating EventStorageBackend (#10224)
3fd3abfb26 Instrument buffer after contract state events stream (#10209)
643a2de006 v2 participant state API draft [KVL-998] (#10210)
5abcd0482e Adds Unit tests for parallel-ingestion [DPP-455] (#10238)
3cdedcf885 kvutils: Extract validators from TransactionCommitter [KVL-1015] (#10235)
00d622f268 Make @UNTIL-LF exclusive, add @UNTIL-LF-FEATURE. (#10240)
2bcbd4e177 es: switch to persistent nodes (#10236)
7f4bc2a472 update compat versions for 1.14.2 (#10233)
4eba4b00e8 Ledger: decouple BD value serialization library from archive library. (#10218)
729afa8c7b Write a few lines on how to run tests against an Oracle database (#10230)
2b67ebb5d4 tf: refactor appr var (#10232)
202b7f7ae7 add akshay to appr team (#10229)
2c5410c229 Release SDK 1.14.2 (#10226)
89d87ad17c update compat versions for 1.14.2-snapshot.20210708.7135.0.aa297840 (#10223)
48393444df TODO Cleanup: QueryNonPruned [DPP-459] (#10225)
0bea5e3ef7 Allow retriable lookupMaximumLedgerTime on contention (#10211)
ca2a9e9bbb Release 1.14.2 snapshot (#10222)
999577a1a7 tweak ES cluster (#10219)
6680d368a1 Warn on DA.Exception import (#10201)
f19f5b0821 LF: Simplify DarReader (#10217)
8f1a8f2390 Release DAML SDK 1.14.1 (#10204)
ebb76dca4c LF: reorganize errors in com.daml.lf.archive (#10213)
69646451f2 Improvements for validation tests (#10214)
5f124c3b64 Avoid collision in execution log postfix (#10205)
9e27ae0d85 Reshuffle cocreature in daml-lf codeowners (#10215)
41b8448b17 LF: Simplify archive reader. (#10208)
38734f02d7 es-feed: ignore invalid files (#10207)
6586dde11e Move exercise context computation to the client side (#10199)
c92c67832f Mark Daml profiler as stable (#10203)
cb1f4ec773 ci/windows: disable spool (#10200)
2326d425bc Publish execution logs on unix platforms (#10194)
42693452ce Drop cleanup of node_modules (#10195)
c8faed8a17 Deduplicate Java codegen running instructions (#10185)
e4c8c390f2 update compat versions for 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10196)
0d881f5e2b Improvements to the documentation with regards to offsets (#10180)
1d5ba4fa42 feed elasticsearch cluster (#10193)
e2bdca6be9 Use PartialFunction for more concise code. (#10191)
fe98c48b65 release 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10186)
4fe626a055 Drop logging on CompletionSource (#10190)
582aa5f08c Fix a typo in an exceptions example. (#10188)
8750c0c47c reduce noise in Scala bindings (#10187)
98b5ffee01 Add divulgence warning and test in script service. (#10179)
8578e56aa7 Tests for transaction validation (#10167)
05a72a3a15 update compat versions for 1.15.0-snapshot.20210705.7286.0.62aabcc4 (#10184)
7b93923c16 [In-memory fan-out] Performance optimizations [DPP-470] (#10127)
6c49619565 Document how to run the Java codegen from the Daml assistant (#10181)
61aa774988 Release SDK 1.15 RC (#10182)
```
Changelog:
```
- [JSON-API] If the service is put behind a proxy filling either of these headers X-Forwarded-For & X-Real-Ip then these will now be respected for logging the request source ip/url
- [Daml Compiler] Importing DA.Exception on LF < 1.14 now produces a
warning that exceptions require Daml-LF >= 1.14.
- [Daml Profiler] The Daml profiler is now a stable feature.
[Docs] Improvements to the documentation with regards to offsets
[Docs] Document how to use the Java codegen using the Daml assistant
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.16 not 1.15
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
Commit log:
```
84e1f3a7f1 participant-integration-api: Move transaction requests to structured logging. [KVL-996] (#10141)
b79d02e28e `ledger-api-bench-tool` - exposed metrics for Prometheus [DPP-471] (#10103)
b61e519a3a Release ledger API test tool per LF version (#10142)
2dfe026cc2 add ES cluster (#10144)
ef9a04caf5 Divulgence crash tests [DPP-433] (#9942)
bb4641756f Daml assistant capitalization (#10140)
a6ee10be9a Move visibility checks into speedy. (#10136)
c764fbe573 contextualized-logging: Introduce different logging types for more structure. [KVL-996] (#10134)
678cab0327 10050 append only schema on oracle (#10051)
f5e5062470 ensure that signatories and observers are disjoint in ledger-api CreatedEvents (#10123)
ffc88d52b4 print version of the ledger api test tool as part of the report (#10119)
621cfa4a91 Resolve contract keys conflicts in disclosed contracts (#9948) (#10034)
f1ffd52e08 Java bindings: add DamlRecord, deprecate Record (#10132)
3df2566305 Scenario: move Scenario Error Throwable to scenario package (#10075)
7521ec9988 [JSON-API] Include contract id in the logging ctx when logging exercise commands (#10131)
e0e333377c update NOTICES file (#10129)
398300b76f LF: Move Speedy Interpretation Error to transaction package (#10091)
d4150ac078 Refactor error reporting in Daml Repl (#10118)
0108fedd4e update NOTICES file (#10112)
6af36fe1e3 Fix Tx.Metadata for normalized transactions (#10108)
7745693648 Remove duplicate index (#10084)
1b617ae22d DPP-428 Add missing indices for index initialization (#10083)
af9382ce92 contextualized-logging: Reduce the API surface and avoid name collisions. (#10102)
01d6770450 json-api perf tests: combine large-ACS frequencies combinatorially instead of in lockstep (#10101)
a44afcff42 Upgrade nixpkgs (#9908)
9498d1509b [DOCS] Let jwt.io link to an already filled in working payload (#10026)
4affb053e9 Force newer version of glob-parent in Navigator (#10105)
f745f10394 LF: Realease LF 1.14 (#10077)
05056ddd8c Cut a flaky expectation some more slack (#10106)
46a66e2f2a Upgrade jest to the latest version (#10107)
84e329885f update compat versions for 1.11.2 (#10104)
a6b536f3c4 Compiler: Make LF 1.13 the default output (#9907)
29ddc88dcb remove redundant spaces in the log format (#10093)
01e329f2de DPP-432 Add exception tests to the JdbcLedgerDao suite (#10040)
3d79cbf2b1 Bump compat versions (#10096)
5bf7d8f570 Fix speedy perf reporting (#10100)
0970821ce7 Bump cpu alloc for build-and-lint (#10099)
cbca779657 [In-memory fan-out] Ensure getTransactionLogUpdates with max fetch size (#10064)
c756153951 Release SDK 1.11.2 (#10094)
3a4235400d rotate release duty after 1.15.0-snapshot.20210622.7213.0.d867d904 (#10086)
ecc2d11536 recommend some daml compiler warnings to enable (#10082)
b2bb45e4d9 Bump daml repl timeout (#10089)
db60d15b5e Log ledger api validation failures at info level (#10080)
f0dc025ac9 Release 1.15 snapshot (#10090)
```
Changelog:
```
- [Ledger API Server] The amount of data logged in the API transaction
service has been reduced at INFO level. Please enable TRACE logging to
log the request data structures.
- [Integration Kit] - ledger-api-bench-tool - exposed metrics via Dropwizard metrics (e.g. for Prometheus)
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
In case a contract key is already present in a past contract in the contract table nullify it. This gets rid of contract keys of previously disclosed contracts. We never discover that disclosed contracts get archived, so we get conflicts on such past keys
[Java bindings] In order to avoid clashing with `java.lang.Record` (introduced
in Java 14), `com.daml.ledger.javaapi.data.Record` has been renamed to
`com.daml.ledger.javaapi.data.DamlRecord`. The old name has been used to
denote a sub-type of the newly renamed one, so it can still be used, but it has
been marked as deprecated.
[Java codegen] The Java codegen now uses the `DamlRecord` type wherever `Record`
was used before.
- [JSON-API] The contract id is now also included in the logging ctx when logging exercise commands
- [LF] Add support for Excepction
- [Compiler] Default ouput LF version is now 1.13
Log ledger api validation failures at info level
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Commit log:
```
adeba20659 DPP-403 log flyway progress (#9964)
401069ef00 prepare for upgrade to Scalaz 7.3.3 (#9997)
3ad9cd58d9 [JSON-API] Log templateId & choice name (if present) on command submissions in the json API (#10005)
af144e8592 LF: Structure package loading errors (#9980)
3583fd1885 [JSON-API] Log source and path of incoming http requests & the response with info level (#10007)
e49c7c8db7 [In-memory fan-out] BufferedTransactionsReader implementation [DPP-413] [DPP-414] (#9882)
acb77336c8 Release 1.14 RC 2 (#10006)
3d2d4b9696 Don't duplicate log failed futures in the http json api (commandservice specific ones) (#9990)
0c32cf2968 Clarify the semantics of DIV_INT64 in LF spec. (#10003)
b79e739196 Fix Type (#10004)
998f106ee6 Drop pkgIds argument from Engine.validatePackages (#9986)
ae5010f3ef [Mutable state cache] Start cache from latest ledger head (#9998)
dd42a5a641 update maven_install_2.12.json (#9999)
66d38cd24d Storage query abstraction first sketch (#9959)
c75013622c Just print out the operator name for the operation logging in CommandService.scala (#9989)
b6faace335 Changing oracle backend from using varray to json array (#9943)
603a4a4b42 Add date to log statements in the HTTP JSON service (#9988)
836a82a6b6 Add proper error handling for missing contract keys (#9972)
85667af7be Address npm vulnerabilities (#9978)
d7d6e37785 LF: Structure Engine Errors. (#9971)
0adbbdcd58 update Daml Hub auth in create-daml-app (#9973)
997a7d449c Use symlinks on Windows (#9969)
e4258e61b1 docs: tweak Exception intro (#9967)
b2c4ef813d rotate release duty after 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9933)
de7a462082 Log call to submitAndWaitRequest with the command id provided in the log ctx (#9940)
630d021331 Collect build-event-stream (#9831)
26e199a719 Rerelease 1.11.2 snapshot (#9965)
8399039a8a Remove broken dep from release job (#9963)
e1db529162 DPP-390 configurable event decoding parallelism (#9931)
cc7af93afc [docs] Remove link to the #daml-contributors (#9928)
f7ac3ba784 Release 1.11.2 snapshot attempt 2 (#9962)
4634147ed1 Adds Indexer state to GRPC health checks [DPP-434] (#9951)
28c567b53d Release SDK 1.11.2 (#9957)
5f0f5756bf Define IncompleteTransaction for use by scenario-service (#9952)
b7fd8338c4 Increase `eventually` timeout when waiting for queue to be killed (#9955)
ca0fb16718 Fix collect_build_data job (#9956)
42030b25ed Skip Scala_2_12 job on releases (#9954)
5e1ddfbd5e LF: Fix specification of ArithmeticError (#9949)
3093a98b87 Forward port additional contract keys validation tests (#9944)
e541558f18 update NOTICES file (#9946)
db792a9774 Proxy ledger API health endpoint in JSON API health endpoint (#9945)
13767b5c5d Load the correct logback file for the http json service respecting the deployment situation (#9938)
4e49cf6814 `ledger-api-bench-tool` - Active contracts stream, completions stream [DPP-398, DPP-399] (#9857)
fce8e54469 release 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9932)
ab55fb2546 Handle module-prefixes in Java codegen (#9929)
7375e28271 Force resolution for css what to 5.0.1 (#9925)
```
Changelog:
```
- [JSON API] The template id & choice name (if present) are now logged on command submissions in the Json API (at trace level)
- [JSON-API] The source and the path for incoming http requests are now logged
- [JSON-API] The http response for a request is now logged
- [JSON API] Errors which arise from the CommandService are not logged twice anymore (thus reducing noise)
- [JSON API] log statements now include the date next to the time
- [HTTP-JSON] Calls which trigger a submitAndWaitRequest are logged with the command id provided in the log ctx
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
- [JSON API] The healthcheck endpoint on the JSON API now proxies the
health check endpoint on the underlying ledger. Previously it only
queried for ledger end to determine ledger health.
[HTTP-JSON]
- fixed that json log output could not be enabled via cli options except via usage of env vars
- [Integration Kit] - ledger-api-bench-tool - reading active contract streams
- [Integration Kit] - ledger-api-bench-tool - Reading completions stream
- [Java Codegen] The Java codegen will now pick up the
`module-prefixes` field from `daml.yaml` which can be used to handle
module name collisions between different DALFs.
```
CHANGELOG_BEGIN
CHANGELOG_END
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
Commit log:
```
adeba20659 DPP-403 log flyway progress (#9964)
401069ef00 prepare for upgrade to Scalaz 7.3.3 (#9997)
3ad9cd58d9 [JSON-API] Log templateId & choice name (if present) on command submissions in the json API (#10005)
af144e8592 LF: Structure package loading errors (#9980)
3583fd1885 [JSON-API] Log source and path of incoming http requests & the response with info level (#10007)
e49c7c8db7 [In-memory fan-out] BufferedTransactionsReader implementation [DPP-413] [DPP-414] (#9882)
acb77336c8 Release 1.14 RC 2 (#10006)
3d2d4b9696 Don't duplicate log failed futures in the http json api (commandservice specific ones) (#9990)
0c32cf2968 Clarify the semantics of DIV_INT64 in LF spec. (#10003)
b79e739196 Fix Type (#10004)
998f106ee6 Drop pkgIds argument from Engine.validatePackages (#9986)
ae5010f3ef [Mutable state cache] Start cache from latest ledger head (#9998)
dd42a5a641 update maven_install_2.12.json (#9999)
66d38cd24d Storage query abstraction first sketch (#9959)
c75013622c Just print out the operator name for the operation logging in CommandService.scala (#9989)
b6faace335 Changing oracle backend from using varray to json array (#9943)
603a4a4b42 Add date to log statements in the HTTP JSON service (#9988)
836a82a6b6 Add proper error handling for missing contract keys (#9972)
85667af7be Address npm vulnerabilities (#9978)
d7d6e37785 LF: Structure Engine Errors. (#9971)
0adbbdcd58 update Daml Hub auth in create-daml-app (#9973)
997a7d449c Use symlinks on Windows (#9969)
e4258e61b1 docs: tweak Exception intro (#9967)
b2c4ef813d rotate release duty after 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9933)
de7a462082 Log call to submitAndWaitRequest with the command id provided in the log ctx (#9940)
630d021331 Collect build-event-stream (#9831)
26e199a719 Rerelease 1.11.2 snapshot (#9965)
8399039a8a Remove broken dep from release job (#9963)
e1db529162 DPP-390 configurable event decoding parallelism (#9931)
cc7af93afc [docs] Remove link to the #daml-contributors (#9928)
f7ac3ba784 Release 1.11.2 snapshot attempt 2 (#9962)
4634147ed1 Adds Indexer state to GRPC health checks [DPP-434] (#9951)
28c567b53d Release SDK 1.11.2 (#9957)
5f0f5756bf Define IncompleteTransaction for use by scenario-service (#9952)
b7fd8338c4 Increase `eventually` timeout when waiting for queue to be killed (#9955)
ca0fb16718 Fix collect_build_data job (#9956)
42030b25ed Skip Scala_2_12 job on releases (#9954)
5e1ddfbd5e LF: Fix specification of ArithmeticError (#9949)
3093a98b87 Forward port additional contract keys validation tests (#9944)
e541558f18 update NOTICES file (#9946)
db792a9774 Proxy ledger API health endpoint in JSON API health endpoint (#9945)
13767b5c5d Load the correct logback file for the http json service respecting the deployment situation (#9938)
4e49cf6814 `ledger-api-bench-tool` - Active contracts stream, completions stream [DPP-398, DPP-399] (#9857)
fce8e54469 release 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9932)
ab55fb2546 Handle module-prefixes in Java codegen (#9929)
7375e28271 Force resolution for css what to 5.0.1 (#9925)
```
Changelog:
```
- [JSON API] The template id & choice name (if present) are now logged on command submissions in the Json API (at trace level)
- [JSON-API] The source and the path for incoming http requests are now logged
- [JSON-API] The http response for a request is now logged
- [JSON API] Errors which arise from the CommandService are not logged twice anymore (thus reducing noise)
- [JSON API] log statements now include the date next to the time
- [HTTP-JSON] Calls which trigger a submitAndWaitRequest are logged with the command id provided in the log ctx
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
- [JSON API] The healthcheck endpoint on the JSON API now proxies the
health check endpoint on the underlying ledger. Previously it only
queried for ledger end to determine ledger health.
[HTTP-JSON]
- fixed that json log output could not be enabled via cli options except via usage of env vars
- [Integration Kit] - ledger-api-bench-tool - reading active contract streams
- [Integration Kit] - ledger-api-bench-tool - Reading completions stream
- [Java Codegen] The Java codegen will now pick up the
`module-prefixes` field from `daml.yaml` which can be used to handle
module name collisions between different DALFs.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
Commit log:
```
3cb8d5c2a7 Deprecate use of divulged contracts (#9930)
e9a2e2e7f2 Allow two different time formats as input for the metrics reporting interval (#9926)
1dc8472017 Fix variable name (#9927)
4037b1cd9e Add metrics to the http json service (#9923)
bd26c43767 LF: fix TransactionCoderSpec (#9922)
69ef6243cb Fix flaky InstrumentedSourceSpec test (#9921)
cbcec32e70 LF: clean up the remaining Exception TODOs (#9913)
537420942d Add dependency proto3-suite for Setup.hs (#9917)
c77324d041 skip fix_bazel_cache on forks (#9920)
ddf93fc413 Address TODOs in JdbcIndexer (#9911)
a0d3e80fd6 Add missing telemetryContext instances in the ledger-api-client's Ctx (#9916)
1e009b4ac2 Upgrade canton to 0.25.0-snapshot supporting lf-1.13 and enable/fix tests (#9915)
db1453640f LF: Freeze archive proto for LF 1.14 (#9912)
15f9dfb963 update compat versions for 1.14.0-snapshot.20210602.7086.0.f36f556b (#9900)
b4247c1511 [In-memory fan-out] BuffersUpdater implementation (#9858)
7e733c3cee DPP-390 remove appendonly todos (#9910)
512f1fd87d LF: Preview version of LF 1.14 (#9906)
2da94c3149 Add warning about using divulged contracts in pruning doc (#9883)
3dee3d042d LF: Fix encoder for Exception (#9904)
b22c04654d ledger-api-test-tool: Only wait for parties on participants under test. (#9903)
b8c02d65b7 Bump perf test for for #9871 (#9902)
4660e57e0c Fix typoed participant name in ExceptionsIT (#9901)
316069d047 Postgres batching refinements (#9898)
7bc925e4d2 LF: Factorize the logic for AST lookup (#9871)
2fa250f06a DPP-387 Remove migration todos (#9896)
65f9395cdd Try to fix windows skipping again (#9893)
850c7310e8 Include PackageMetadata in LF interface (#9892)
41266c35e2 Fix the formatting on the LF spec. (#9889)
dc79830239 DPP-393 Enable sandbox tests for the append-only schema (#9876)
3c83b77dd8 Parallel Ingestion renames (#9894)
6e48034edc Dpp 392 todo cleanup postgre batch (#9875)
08ed7b61ff rotate release duty after 1.14.0-snapshot.20210601.7081.0.7d716e6d (#9878)
081c54b411 Revert "Try to unflake JSON API failure tests (#9855)" (#9887)
df0f3ff7b0 maybe fix deleted release tags (#9891)
4b020dddc3 Release ledger-api-bench-tool (#9890)
d80191435f Release new snapshot (#9885)
4a4dde0f19 Implemented JdbcLedgerDaoTransactionLogUpdatesSpec (#9813)
```
Changelog:
```
- [Ledger API] Use of divulged contracts in later transactions is
deprecated. Divulged contracts were already incompatible with
pruning. We are working on features to handle the cases currently
covered by divulgence.
- for applications which support the --metrics-reporter-interval cli option, these now support both the java and scala duration string format (e.g. PT1M30S and 1.5min)
Small fixes to the doc
[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)
[ledger api] Add warning about incompatibility between pruning and
divulged contracts to pruning docs.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@cocreature is in charge of this release.
Commit log:
```
7d716e6dba Try to unflake JSON API failure tests (#9855)
2d49939360 Fix the notion of transaction equivalence in the ledger model for exceptions. (#9850)
fd8f8a6ad9 Daml-lf suffixCid ignores V0 contract ids rather than erroring (#9867)
e38bb4a55d Make data-deps exception tests version-aware. (#9870)
94129b2254 Add tests for decoding of UpdateTryCatch (#9865)
1cc0906267 Adda test for an unhandled builtin exception to ExceptionTest.scala (#9866)
d7c60f5258 resources: Only mutate test mock objects inside `synchronized` blocks. (#9869)
3a0b2f9ccc remove 2x 8020 todo tags (#9868)
1595f06215 Special case _tryCatch @Update in LFConversion. (#9864)
c0359ac9ae Add trace context propagation in CommandClient.trackCommandsUnbounded [KVL-961] (#9833)
2a1e1ba1d5 Add a Foldable instance for Set (#9860)
590b8b3367 Reshuffle tests for AnyExceptionMessage (#9863)
89bd478949 Reenable AnyException tests in ExceptionTest.scala (#9861)
1573984673 Include rollback nodes in danglingRefGenNode (#9851)
e5d4553208 Remove flyway warning when using the append-only schema (#9840)
a4e33c9e1b Add metrics to parallel indexer (#9841)
972af2a1ae Test To/FromAnyException primitives in DecodeV1Spec (#9852)
5f4f019f49 update NOTICES file (#9859)
f5f2a8a3f7 Remove obsolete transaction todo (#9853)
8fc45432d1 Uniform configuration of the command service across all ledgers (#9839)
1f4a958432 ledger-api-bench-tool - single-value metrics (#9822)
6a1c7974dd ledger-api-bench-tool: TLS support (#9824)
55283472df Test TransactionsReader.getContractStateEvents within the JDBC DAO suite (#9849)
6af6c930a3 Fix behavior of Math.sqrt at 0 (#9818) (#9828)
7855ea50c2 Switch to Ubuntu 20.04 nodes for ghc lib (#9838)
c0fccfc671 Switch Scala version Bazel config (#9848)
e033277804 Switch to our own nodes for releases (#9845)
b6a89a6948 Bump ws to address security advisory (#9844)
25b7e544ce Only resolve template id once (#9837)
a645971e72 Enable mutable-contract-state-cache for ledger-on-SQL append-only tests (#9836)
dcd33a7b4b Add a section on exceptions to the Daml intro (#9832)
859cae3b46 [Mutable contract state cache] Tests for contract/key state new methods in ContractsReader (#9825)
ffe55863ff Track oldChild in PartialTransaction context (#9815)
6e17e2de75 [In memory fan-out] Transaction log updates stream implementation (#9792)
a0373c4855 Ledger model section on transaction normalization. (#9816)
f355931a59 Fix typeclass detection in data-dependencies (#9830)
9626a5952d Speedy: Clean SBuiltin (#9829)
269237ea76 Fix authorization in try contexts (#9823)
cb06fb3448 `ledger-api-bench-tool` - min consumption speed SLO [DPP-401] (#9808)
0868324c24 update NOTICES file (#9827)
a166e79aca Added maxlength information to id string (#9826)
63bc0d1a96 EventsBuffer implementation for in-memory fan-out for Ledger API serving (#9775)
0c12259c04 Speedy: Use SAny instead of SAnyException to reperesent AnyExcpetion (#9819)
6caa61c9d4 participant-state: rejection reason abstractions (#9764)
63c471fa79 Upgrade protobuf-java and scalapb. [KVL-938] (#9713)
f177c1a6e4 `ledger-api-bench-tool` - Moved classes to individual files [DPP-401] (#9803)
a3b10dc6f9 LF: fix parser (#9809)
9f55da813f update NOTICES file (#9812)
6846dbfaf8 update compat versions for 1.14.0-snapshot.20210526.7024.0.aedb9a82 (#9806)
1f021b25c6 LF: Drop Builtin Exceptions completly (#9790)
ea5b710d7d rotate release duty after 1.14.0-snapshot.20210525.7017.0.2710fad0 (#9795)
0c7b149068 Add reference docs for exceptions (#9807)
6b67ba06f2 LF: SBAnyExceptionMessage queries for unknown packages. (#9804)
fdab1e68d7 Parallelize daml-assistant integration tests. (#9802)
2e9c56e52c DPP-393 Activate JdbcLedgerDao suite for the append-only schema (#9793)
274fcaee10 release 1.14.0-snapshot.20210525.7017.0.2710fad0 (#9801)
3241ad6a41 `ledger-api-bench-tool` - updated metrics reporting model [DPP-400] (#9749)
```
Changelog:
```
Modify command line options of the ledgers based on the kvutils/app library:
- Remove max-commands-in-flight sub-option from the participant group.
- Add command service configuration options in line with sandbox and daml-on-sql: max-commands-in-flight, max-parallel-submissions, input-buffer-size
- [Integration Kit] - TLS support for the ledger-api-bench-tool
[Daml Compiler] Ensure that sqrt(0.0) == 0 and 0 ** 0 == 1
- [Integration Kit] - ledger-api-bench-tool - new consumption speed service-level objective for the ledger-api-bench-tool, configurable with a CLI parameter --min-consumption-speed
- Participant-state: Introduced versioned form of rejection reason
- [Integration Kit] - ledger-api-bench-tool - new parameter max-delay for specifying the service-level objective for max delay
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@SamirTalwar-DA is in charge of this release.
Commit log:
```
a6c7b86afc LF: Drop support of TO_QUOTED_TEXT_PARTY (#9721)
9d23b4337f Split DA.Internal.Exception.Types into three packages. (#9709)
66d039eb5b Upgrade gatling to version 3.5.1 (#9714)
ac9b8adc5c TODO Cleanup: store.backend package (#9693)
679ce3b7d3 LF: Release LF 1.13 (#9705)
73c479310d update NOTICES file (#9711)
f249eec24f drop per-commit compatibility_macos (#9730)
55c36cc93e Compiler: drop BEToTextBigNumeric in favor of BEToText (#9728)
2a2c745978 Implement name collision check for type synonyms. (#9723)
c7cc1bfdfa KVL-914 Add more unit tests for tracing (#9708)
e63e0cfb77 Upgrade canton to 0.24.0 (#9722)
0a1d00d1b3 Canonicalize junit output path (#9710)
d4fca0305b LF: Change error exception for Arithmetic builtins (#9692)
1e5664054d Release: Switch Andreas and me around in the release rotation. (#9719)
e50885caa8 Dpp 388 todo cleanup parallel indexer (#9690)
fc745f2224 LF: clean shifting BigNumeric builtin (#9704)
7a97d88f42 DA.BigNumeric: Split shift, add roundToNumeric. (#9702)
b1738c7202 Switch Scala default to 2.13 (#9699)
1edb110f99 Fix simplifier safety for AnyExceptionMessage (#9707)
0ab37b3774 Use new exception builtins in stdlib. (#9703)
7d472e178d Optimistically increase Patience for fixing flakiness in MutableCacheBackedContractStoreSpec (#9697)
69f51e4b5e speedy: Compile new AnyException primitives (#9700)
3666cf2f54 KVL-914 Add unit test and fix trace context propagation from config management service (#9694)
4e63299919 Simplify kvutils contract keys validation (#9628)
010e2b1b27 Fix client_server_build/test with port file (#9701)
6d9490c0bb Never create a rollback node containing pre-exception descendants. (#9679)
b19c8f2a7c Drop sanitize, add MagicHash in data-dependencies. (#9698)
c669a00883 Release SDK 1.13.1 (#9696)
395ff58eec Damlification of Bazel files (#9670)
9b53251c74 Differentiate between negative inputs & errors in contractKeyInputs (#9683)
cd9933328a runtime: Damlification of Scala files (#9668)
fb0551bdfd Fix blackduck flag names (#9695)
4b3b9ef165 Add race condition tests for exceptions (#9688)
927242be40 KVL-914 Add and rework unit tests for tracing (#9686)
6568336588 Lift constraint tuples up to a type synonym in data-dependencies. (#9687)
80f65b4734 LF: clean compilation of SToTextXXX builtins (#9682)
ed14f97528 Move race condition utilities to test tool infrastructure (#9685)
e5646a9c73 update compat versions for 1.13.1-snapshot.20210512.6834.1.5f532380 (#9681)
6629aab009 DEL-8479 tag LF libs for SDK 1.13 release (#9680)
55abd0b7f0 Add a data-dependencies test for user-defined exceptions (#9659)
142767c421 Split daml test-script code into a separate library (#9674)
d436686173 Rerelease 1.13.1 snapshot (#9678)
5d181c6ef4 Throw on internal errors instead of setting ptx to aborted (#9654)
ac2e285ed3 Release 1.13.1 snapshot (#9675)
af7f88fca5 restore natural join and avoid table aliasing in participant contracts queries (#9671)
625592e7b5 Rename CorrelationID to InstanceUUID as it has been used wrongly. (#9672)
638547fdde rotate release duty after 1.13.0-snapshot.20210511.6892.0.ca9e89b3 (#9648)
c282a09ffb KVL-914 Expose metrics-test-lib for the oem integration kit (#9662)
cc01e933e5 Update error message in interface reader. (#9657)
2dc8b8f1de Remove version field from rollback node. (#9627)
b58d30b9e3 skip Windows tests on release (#9656)
40f69ee5af fix release PR notifications (#9658)
691915353c Fix update metrics population in parallel indexer (#9655)
931274a14e Drop version check from TransactionPreprocessor (#9651)
4620851ba2 release 1.14.0-snapshot.20210511.6892.0.ca9e89b3 (#9647)
0d09db5c48 Update date on code owners in RELEASE.md (#9652)
e61e6b22e0 Release SDK 1.13.0 (#9650)
9e6d6f7cf0 Support append-only schema in Sandbox migration tests (#9637)
fb9a196431 update NOTICES file (#9649)
58d4281743 Document web socket event handlers of TS Stream type. (#9634)
111e1d3798 Fix sandbox classic package upload (#9633)
```
Changelog:
```
- LF: make LF 1.13 stable
Add support for BigNumeric
- Ledger API: Bump Ledger API version for LF 1.12
- [Daml Standard Library] `DA.BigNumeric.shift` has been split into
`DA.BigNumeric.shiftLeft` and `DA.BigNumeric.shiftRight`.
`DA.BigNumeric.roundToNumeric` is introduced, for rounding and
converting a BigNumeric to a Numeric in a single move.
- [Daml Compiler] Fixed a data-dependencies bug where functions in a data-dependency that used a constraint tuple constraint (e.g. `Template t`) could not be directly invoked.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
The first one ended up with a corrupted Windows installer. I couldn’t
find anything in the logs indicating why it would be broken and this
is the first time I’ve seen this so hopefully just rereleasing fixes it.
changelog_begin
changelog_end
* release 1.13.0-snapshot.20210511.6892.0.ca9e89b3
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@victormueller-da is in charge of this release.
Commit log:
```
ca9e89b3da check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
4a460973a2 Block concurrent SDK installations. (#9640)
c757b61516 Check byKey in transaction validation (#9641)
4f0ceff7a2Fix#7170 (#9618)
d3106682dd Add a REPL for each Scala target, for debugging. (#9643)
22b36b0b01 Include byKey in LF transaction spec (#9642)
ca027e3dde Add Transaction.contractKeyInputs to inputs of a transaction (#9631)
c3d79d4656 Allow OverloadedLists in data-dependencies (#9636)
f8b35fef9c Cleans stateUpdates of ReadWriteServiceBridge (#9630)
3750b16a11 Serializalize byKey flag in Daml-LF transactions (#9632)
3b59f8ce87 ledger-api-bench-tool - prototype [DPP-370] (#9606)
f7b33d93fb Authorization for daml script exports (#9629)
0d1f3db8a4 LF: refactor builtin exceptions in Speedy (#9605)
d6d01b04ef Swap order SEScopeExercise and SBUBeginExercise (#9621)
6c919b3877 sandbox-on-x: Support `beginAfter` in state updates. (#9624)
26a8011ba5 TLS for Daml Script exports (#9626)
9242540022 Make error throw a GeneralError. (#9613)
bdcf0ecb66 update NOTICES file (#9623)
4c1fbeb194 Add duplicate contract key checks to Speedy (#9607)
871279f3a6 LF: extends LF command with Fetch and Lookup (#9587)
0acc4f115c Patch old Bazel derivation (#9622)
b082274885 Address vulnerabilities in Navigator (#9617)
560653a5d7 Unwind partial transaction on mustFail (#9608)
f51145cb88 Support mapping from old to new party ids in daml exports (#9591)
89e46bf90b Address security vulnerabilities in //compiler/daml-extension (#9615)
fd62671e0b Introduce SINCE-LF-FEATURE in integration tests. (#9616)
48939b513b Address vulnerabilities in //language-support/ts/packages (#9614)
cf59246d44 Add support for daml exceptions for append-only indexer (#9609)
ab29f7c07c Move activeness check of globalKeyInputs into archive (#9610)
5c28de36cb Upgrade grunt to address cve (#9611)
22ba5fddd2 Remove builtin exception types from protobuf and ASTs. (#9595)
b09a95fb6f Adapt indexer to empty divulgences (#9598)
2fc7489e44 Filter divulgence to an empty set of parties (#9600)
ad45213b66 participant-integration-api: Avoid the serial EC in integration testing. (#9603)
f742a4334e Dpp 336 sandbox classic on append only (#9561)
e68bc0dff0 Mark reset service tests as flaky (#9602)
2176173ba2 Remove 1.dev-only things from LF 1.13 protofile. (#9599)
03034ec3bf DPP-359 Add indexer flow level benchmark (#9509)
5aa54017eb Drop todo for conversion of exception primitives (#9597)
968b5d8d39 KVL-921 Expose opentelemetry Context from Telemetry and TelemetryContext (#9573)
112c387e5a Refactor out setGlobalLogLevel into ContextualLogging (#9592)
e584fecd3c Drop com.daml.lf.engine.Event.collectEvents (#9596)
80b07da309 Only archive a key if it was brought into scope before (#9546)
7ea9340d64 Support old bash in daml-sdk-head (#9593)
45fbdefb06 Handle rollback nodes in protoNodeInfo (#9589)
42d189dbbe Support exceptions in Daml-LF encoder (#9590)
9c3913a3d6 LF: SBUKeyBuiltin clean up. (#9572)
5128206ebf Ledger API Server: Named threadpools (#9588)
26a53d8469 Add logging command line option to ledger http service (#9581)
409833f850 Address rollback todo in ancient migration (#9583)
a27b2c56bc Address more exception todos (#9582)
2040556dbb Support rollback nodes in replay adapter (#9584)
192a77a6d8 update compat versions for 1.13.0-snapshot.20210504.6833.0.9ae787d0 (#9575)
81d508b775 LF: make SBuiltins pure (#9580)
de80a6dc60 Drop ValueBuiltinException (#9576)
1c456be79d Release yet another 1.13 snapshot (#9574)
800989145b Disable migration tests on macos on PRs (#9571)
2e93de79a4 Fix recording exceptions in Spans, add unit tests [KVL-874] (#9544)
121ded3565 Accept a comma-separated list of parties for daml ledger export (#9568)
```
Changelog:
```
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns
- [Integration Kit] - Created the ledger-api-bench-tool prototype for benchmarking ledger transaction streaming capabilities
* [Daml export] Users can now define a mapping from parties in the
original ledger state to parties to be used when recontructing the
ledger state. The ``parties`` component of the argument to the
generated export script now takes a mapping from old party name to new
party name.
- [Ledger HTTP Json Service] Logging now also tells service name if log level was changed.
- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`
```
CHANGELOG_BEGIN
CHANGELOG_END
* Release snapshot 1.14.0
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: victor.mueller@digitalasset.com <mueller.vpr@gmail.com>
Gary Verhaegen
Moritz Kiefer
azure-pipelines[bot]
Stefano Baghino
Victor Peter Rouven Müller
tudor-da
Kamil Bożek
mziolekda
Andreas Herrmann
nickchapman-da
Remy