This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@adriaanm-da is in charge of this release.
Commit log:
```
cfcdc13c6e Add deprecation output for sandbox classic (#11119)
ac5c52dbe4 DPP-609 Add StorageBackendTestsDeduplication (#11088)
55af7ad729 HttpServiceTestFixture provides a sandbox-classic ledger used for performance tests (#11128)
63ab3f3e50 Fix change ID references in services.rst (#11130)
31db15d555 Port error code changes (#11113)
9fd8182bbb move reusable functions from ContractsFetch to new fetch-contracts library (#11057)
c2084f6e14 Try and improve the explanation of compatibility test exclusions (#11114)
4df9b7c8f1 Rename command completion dedup time to duration in participant code [KVL-1057] (#10901)
11bc22d085 LF: builtins to create, signatory, and obersver on interface payload. (#11120)
d4cb1f945e KV ledgers: disable participant side command de-duplication [KVL-1083] (#11095)
b9a5a83175 Minor: fix trace logging in `TimeBasedWriteSetSelector` (#11117)
018e90854c [JSON-API] Make key_hash idx non unique (#11102)
429f43731a Remove HA indexer integration test on H2 (#11104)
9cffa1faf6 LF: check LF transaction protobuf aginst local snapshots (#11064)
d7ee278518 Optional table prefix for trigger service (#11047)
268b2d3da0 Command deduplication - test case that uses CommandService and CommandSubmissionService [KVL-1106] (#11098)
fac05f657b bypass value enrichment for the performance test (#11112)
4860271a53 Port file was not written to... where? (#11108)
a8d703b996 Increase akka server request-timeout for HttpServiceIntegrationTests (#11109)
37d94aaef6 Add unit for daily performance tests (#11107)
eb4ac8a397 Increase timeouts for resetting database (#11103)
db75f7da0f interface methods: Scala Typechecker (#11097)
d3e6f16c61 [JSON-API] Migrating tests to use sandbox next (#11034)
ffc8d6894f Data dependencies deserialize imports (#11037)
4b31bf088f update compat versions for 1.17.0 (#11075)
9fd6326ae7 interfaces: consuming/non-consuming iface choices (#11009)
229ce4776f Data dependencies serialize imports (#11038)
5d9ec65cc6 Add test case for parallel command deduplication using mixed clients [KVL-1090] (#11093)
d64d965230 kvutils: Print state updates before and after normalization. (#11096)
be216aa872 update NOTICES file (#11089)
517e866df5 Extract common code for command dedup conformance tests [KVL-1090] (#11092)
7a4963b43d add `--max-inbound-message-size` flag to `daml ledger export` (#11087)
26d10b267d Drop NodeId type parameter (#10921)
3cbd9229aa More NonEmpty functions (#10930)
bcd468625c interface methods: Speedy (#11076)
85adaab550 [DPP-417][DPP-595] Error codes switching - follow-up (#11074)
d7b280b60c Remove debug print in daml-lf/interpreter build (#11091)
1ed6428980 LF: move archive snapshots in a separate directory (#11081)
5e424f8d1c Command dedup conformance suites readme (#11051)
8290347f74 DPP-496 HA indexer integration test (#11033)
735c3090a3 Switch from InputStream to Byte Array at the binary content JDBC transport (#11072)
6ae3afa8fb fix perf reporting (#11073)
bf801a6e1b Fix SimpleDeduplicationBasic when participant deduplication is disabled (#11083)
62234dc763 [Short] Remove unused code (#11079)
c1d1521a14 interface methods: Scala AST (#11070)
7dd9c2d3f0 Remove expectations for internal failures from parallel command dedup tests (#11061)
51125991e5 Explicit discard in daml-lf/interpreter (#11067)
554b36c817 rotate release duty after 1.18.0-snapshot.20210928.7948.0.b4d00317 (#11059)
d156964f0c Release SDK 1.17.0 (#11062)
6d8cf7089c Handle interface methods in LF conversion (#11054)
87ecf1fe63 daml-ledger: better errors for non-json responses. (#11055)
ec2d26f31f [Divulgence pruning] Fixes divulgence pruning offset update query (#11046)
f13c6d61ec release 1.18.0-snapshot.20210928.7948.0.b4d00317 (#11058)
d1805a3000 More dependabot fun (#11063)
07b273aab5 update NOTICES file (#11060)
```
Changelog:
```
- [JSON-API] make key_hash indexes non-unique, this fixes a bug where a duplicate key conflict was raised on the query store when the same contract was being witnessed twice by two separate parties
* [Trigger Service] Enable the new `tablePrefix` setting in the `--jdbc`
flag to add a prefix to all tables used by the trigger service to
avoid collisions with other components using the same db-schema.
* [Trigger Service] Enable the new ``--allow-existing-schema`` flag to
initialize the trigger service on a database with a pre-existing
schema.
- [Daml Assistant] Add support for `--max-inbound-message-size` flag to the Ledger Export tool.
Add cons, find, delete and deleteBy functions to NonEmpty module.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
Commit log:
```
b4d00317b4 detect unsynchronized contract table and retry (#10617)
3d779cfe12 [Mutable state cache] Fix initialization offset (#11024)
5458aa890c Switch on NonUnitStatements warning in daml-lf/transaction (#11048)
9641fd5f83 auth middleware: no print secret (#11050)
a885f52c4d [DPP-417] [DPP-595] Add error code version switching mechanism (#11035)
12e0c72d5c fix blackduck logic (#11049)
eb87b3439b kvutils: Add the logging context for ledger state operations. (#11030)
b7daa5f7d8 Address remaining dependabot alerts (#11045)
5e43f8c703 es: drop jobs-* indices (#10857)
03203b7360 Define encoding/decoding for module imports (#11036)
57a15972c7 Setting timeoutToleranceMillis to 10 minutes to prevent flakiness (#11043)
df59f3fe8e Fix Navigator dependabot alerts (#11044)
6bf45a344a Upgrade Navigator to Webpack 5 (#11040)
80e217e11f [DPP-622] Add conformance tests that verifies TLSv1.0 and TLSv1 are disabled. (#10983)
626e1fbd7d Small lf value.cids regression fix (found by canton unit tests) (#11032)
a4629a4450 KV: Ignore daml_lf_1.proto when checking for KV protobuf compatibility (#11021)
ee9be65d68 kvutils: Add metadata to `Err` [KVL-1032] (#10992)
0d3ae6e14c interface methods: Haskell Typechecker (#11028)
e36eb46f59 Resolve `set-value` to 4.0.1 and above (#11029)
4075624cfc interface methods: Haskell AST (#11018)
7c1fd50469 Bump grunt-browserify (#11026)
5f3f5824f2 Upgrade webpack-dev-server in Navigator (#11025)
91be1e1659 Drop matchdep dependency from docs build (#11023)
fe9aeffeaf Increase es disk size (#11019)
eac7963b87 LF: Refactor ProtoTest.scala (#11020)
e79a30aa80 For the client binding propagate the full original completion [KVL-1112] (#10879)
59ad995282 fix buf check (#11014)
abc3e6691c Increase the tolarance for handover of control in HaCoordinatorSpec (#10997)
19b2bf477f LF: Cosmetic clean-up in the Speedy Compiler (#11015)
cb0e41f101 LF: Add interface support to the Preprocessor (#11013)
c33297c5fe Remove `transactionNormalization` flag. (#11010)
f5d2135f97 Check protobuf compatibility of `main` and PR commits w.r.t. previous stable release [KVL-1109] (#10950)
bf8b75dab5 interface methods: Add protobuf definitions. (#11005)
88e1430a42 Make LargeTransactionTest use ValueEnricher, so it can work with normalized transactions coming out of the engine. (#11003)
a043926c30 rotate release duty after 1.17.0-snapshot.20210921.7889.0.1b473c2b (#10972)
35666caae8 Add MINIMAL pragma for Additive type class (#11001)
8de162baec [DPP-586] Upgrade to netty 4.1.67.Final and netty-tcnative-boringssl-static 2.0.40.Final (#10956)
871d03ba65 release 1.18.0-snapshot.20210922.7908.0.ced4a272 (#10998)
721575ea73 [JSON-API] Postgres perf job (#10986)
f2d9f07417 Release RC2 for SDK 1.17.0 (#10996)
```
Changelog:
```
- [JSON API] Under rare conditions, a multi-template query backed by database
could have an ACS portion that doesn't match its transaction stream, if
updated concurrently. This conditions is now checked and accounted for.
See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
- The OAuth2 Middleware now obfuscates its Client Secret when logging
its config.
- [Integration Kit] We have added ``loggingContext`` as an implicit
parameter to more _kvutils_ trait methods. Implementors may need to do
the same in their trait implementations. This can make it easier to
log with the appropriate context.
java-client-bindings - the original full completion is included in the `CompletionResponse` when available
Daml on SQL, Integration Kit, Sandbox: Drop support for TLS 1.0 and 1.1 in Ledger API.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
@garyverhaegen-da is in charge of this release.
This supersedes the failed #10971 and #10993.
Commit log:
```
ced4a27240 Include concurrency info in output (#10970)
a0b6800a93 Limit maximum number of concurrent tests (#10969)
a0fe0f6239 release 1.18.0-snapshot.20210921.7889.1.1b473c2b (#10993)
58c82b6e41 LF: reduce usage of NodeCreate.coinst (#10988)
f56ce2a757 kvutils: Add structured error data to the error metadata [KVL-1032] (#10964)
3f4dbf6f8e interfaces: Syntax highlighting for interfaces (#10987)
d52a2857da interface: Add test for conversion functions (#10991)
28b8d9a1f7 bump dotnet (#10979)
fe10ffbe8f Use ValueEnricher in ScenarioRunner. (#10897)
bdad7d5c99 Upgrade Java on Windows (#10965)
34390f7062 interfaces: Implemented conversions, added test. (#10982)
7740b60836 participant-integration-api: Improve the CommandConfiguration docs. (#10978)
04f322e2aa Bump resources for daml-ledger tests (#10984)
6d9922e1f1 Retry upsert of command deduplication on oracle and h2 [DPP-609] (#10976)
eb4f1b2a05 LF: Move lookup tests out of EngineTest (#10973)
99836d2610 Handle fetchByKey callback correctly in scenario runner (#10980)
23e6a2dddb Improve feedback from failing match in AuthMatchers (#10981)
e299103450 [Short] Small test tool's CLI parser refactor (#10923)
fc153a91b2 release 1.18.0-snapshot.20210921.7889.0.1b473c2b (#10971)
1b473c2b9a Switch trigger service time provider default to wall-clock time (#10967)
4acf34b5fc Add conversions from/to interfaces in Daml parser (#10954)
d50df11f77 interface: Add to/from interface in scala ast (#10960)
9f88e0919f Drop jdk8 override (#10963)
e0587bcdd2 Add a bit of detail to RegisteredTemplates documentation, motivated by: (#10962)
cebe6ed482 Refactor safetyStep in the simplifier. (#10948)
ac192fc6db interfaces: Add to/from interface in Haskell typechecker (#10951)
4b4d7a3af7 Remove update normalizer which was too aggresive (#10925)
66e1098070 Add tests for the `CompletionFromTransaction` converter [KVL-1104] (#10885)
724e50d739 interfaces: Add to/from interface in Haskell AST (#10945)
9a8d55ad13 Change slack link to discuss wiki. Fixes#10946 (#10947)
8b3b033fad LF: Test preprocessor resuming (#10936)
2edfc062a0 ifaces: name collision, typecheck fetch/exercise (#10896)
5dc15c6911 LF: rename language Interface to PackageInterface (#10938)
054c6abb63 Upgrade canton to a more recent version (#10944)
b8533d50e9 [JSON-API] Production/HA documentation. (#10903)
a33176265a Clarify `CommandTracker` [KVL-1104] (#10943)
5244643d16 Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
6cc42ee9f4 rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
88ef05e557 sandbox-classic: Only allow `--max-parallel-submissions` here. (#10941)
ac02dbdeb9 LF: Exhaustive test for valueTranslator. (#10927)
409c0b4f60 interfaces: Add to/from_interface in proto (#10937)
02c8a9dfb8 Split CommonStorageBackend (#10871)
dc71a6aea8 update NOTICES file (#10932)
0ba54a4bf9 Add a missing test case to the `CommandTrackerFlowTest` (#10939)
855ecdf1a7 [DPP-572] Add ledger API test case for verifying conformance to `--min-tls-version` flag. (#10898)
3e13e3d87e Switch to stable urls in scoop python manifest (#10933)
fc2c87d950 ledger-api-client: Generate a submission id if it's empty in the `CommandClient` [KVL-1104] (#10926)
04d8f753b2 Clean up sandbox-on-x conformance tests. (#10766)
b4541b5a4d Logging delay of submitted commands (#10912)
b8e21d8f48 Fix takeFilter for the test StreamConsumer (#10918)
906368d7e2 LF: exhaustive test for CommandPreprocessor (#10914)
61d214e451 Add fetch, exercise implementations for interfaces in speedy. (#10911)
d01f8e1c35 Fix flaky ApiConfigManagementServiceSpec test (#10922)
50291ed61b interfaces: scala typechecker implementation (#10867)
cac8391d4c Bump ghc-lib to include daml interfaces parser (#10747)
b6a6bf7aac [Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
308f938512 Dpp 494 unit testing ha coordinator (#10862)
30f74adc99 Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
9582e019ec LF: Refactor PreprocessorSpec test (#10909)
9b0fa29aec Separate exercise & fetch for interfaces from templates (#10908)
f4adee91ca Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
8a391189a6 Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
8e22bb6b2d Drop ContractId typeparameter from Value (#10827)
45437059d4 Drop cocreature as a codeowner from runtime things (#10906)
e6e81470bb Release 1.17.0 RC from second to last commit (#10904)
f08ac5f65c Desugar interface implements declarations (#10895)
b5648c0e3d Make `CommandTracker` distinguish submissions of the same command using `submissionId` [KVL-1104] (#10868)
```
Changelog:
```
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
```
CHANGELOG_BEGIN
CHANGELOG_END
The `main` branch build for #10971 failed. #10979 should have fixed it,
so let's try again.
This needs to be merged _after_ #10979.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.17.0-snapshot.20210921.7889.0.1b473c2b
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
Commit log:
```
1b473c2b9a Switch trigger service time provider default to wall-clock time (#10967)
4acf34b5fc Add conversions from/to interfaces in Daml parser (#10954)
d50df11f77 interface: Add to/from interface in scala ast (#10960)
9f88e0919f Drop jdk8 override (#10963)
e0587bcdd2 Add a bit of detail to RegisteredTemplates documentation, motivated by: (#10962)
cebe6ed482 Refactor safetyStep in the simplifier. (#10948)
ac192fc6db interfaces: Add to/from interface in Haskell typechecker (#10951)
4b4d7a3af7 Remove update normalizer which was too aggresive (#10925)
66e1098070 Add tests for the `CompletionFromTransaction` converter [KVL-1104] (#10885)
724e50d739 interfaces: Add to/from interface in Haskell AST (#10945)
9a8d55ad13 Change slack link to discuss wiki. Fixes#10946 (#10947)
8b3b033fad LF: Test preprocessor resuming (#10936)
2edfc062a0 ifaces: name collision, typecheck fetch/exercise (#10896)
5dc15c6911 LF: rename language Interface to PackageInterface (#10938)
054c6abb63 Upgrade canton to a more recent version (#10944)
b8533d50e9 [JSON-API] Production/HA documentation. (#10903)
a33176265a Clarify `CommandTracker` [KVL-1104] (#10943)
5244643d16 Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
6cc42ee9f4 rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
88ef05e557 sandbox-classic: Only allow `--max-parallel-submissions` here. (#10941)
ac02dbdeb9 LF: Exhaustive test for valueTranslator. (#10927)
409c0b4f60 interfaces: Add to/from_interface in proto (#10937)
02c8a9dfb8 Split CommonStorageBackend (#10871)
dc71a6aea8 update NOTICES file (#10932)
0ba54a4bf9 Add a missing test case to the `CommandTrackerFlowTest` (#10939)
855ecdf1a7 [DPP-572] Add ledger API test case for verifying conformance to `--min-tls-version` flag. (#10898)
3e13e3d87e Switch to stable urls in scoop python manifest (#10933)
fc2c87d950 ledger-api-client: Generate a submission id if it's empty in the `CommandClient` [KVL-1104] (#10926)
04d8f753b2 Clean up sandbox-on-x conformance tests. (#10766)
b4541b5a4d Logging delay of submitted commands (#10912)
b8e21d8f48 Fix takeFilter for the test StreamConsumer (#10918)
906368d7e2 LF: exhaustive test for CommandPreprocessor (#10914)
61d214e451 Add fetch, exercise implementations for interfaces in speedy. (#10911)
d01f8e1c35 Fix flaky ApiConfigManagementServiceSpec test (#10922)
50291ed61b interfaces: scala typechecker implementation (#10867)
cac8391d4c Bump ghc-lib to include daml interfaces parser (#10747)
b6a6bf7aac [Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
308f938512 Dpp 494 unit testing ha coordinator (#10862)
30f74adc99 Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
9582e019ec LF: Refactor PreprocessorSpec test (#10909)
9b0fa29aec Separate exercise & fetch for interfaces from templates (#10908)
f4adee91ca Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
8a391189a6 Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
8e22bb6b2d Drop ContractId typeparameter from Value (#10827)
45437059d4 Drop cocreature as a codeowner from runtime things (#10906)
e6e81470bb Release 1.17.0 RC from second to last commit (#10904)
f08ac5f65c Desugar interface implements declarations (#10895)
b5648c0e3d Make `CommandTracker` distinguish submissions of the same command using `submissionId` [KVL-1104] (#10868)
```
Changelog:
```
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.18 not 1.17
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Manual release process. @stefanobaghino-da is in charge of this release.
Commit log (since 1.17.0-snapshot.20210907.7759.0.35a853fd):
```
976ca400eb Force JSON API to refresh packages for GET requests to /v1/query (#10835)
e42cd3a1c6 LF: add interface to scala AST and decoder (#10830)
07b72d0f3e Update Ledger API's proto documentation [KVL-1094] (#10815)
648acf52f8 Fix typo in app-arch.rst (#10798)
dc9fbb5aa9 Stringify errors in create-daml-app before calling alert (#10831)
90b008ef0d Stop using `controller … can` syntax in ledger tests (#10814)
e490b69471 Clarifications of code ownership in /ledger and /ledger-api (#10742)
8cdeb7539a make auth middleware aware that actAs => readAs (#10825)
e263d43aca Ledger API test tool: test that record time is monotonically increasing [KVL-1053] (#10552)
e5793a317b Account for multi participant conformance tests when updating the config [KVL-1092] (#10788)
2e39d7b647 Deprecate `Commands.deduplication_time` field [KVL-1096] (#10802)
7824201b8d Get rid of extra new line and prune precision for all gatling values (#10820)
a03f52a15e Add ledger API tests about command deduplication information in completions [KVL-1057] (#10748)
61a07b1986 Add command line option max-deduplication-duration for sandbox and KV [KVL-1098] (#10816)
0800504955 Fix DuplicateSubmissionID test exclusions (#10819)
1ff34ffbe8 Remove workaround for akka supervisor issue (#10818)
6d347d4e88 Add daml interfaces to the Haskell LF AST (#10811)
cecb27fe16 Stop using controller can syntax in skeleton template (#10799)
8830eaa2ec Drop controller can syntax from compiler tests (#10807)
9038a8051b Update code documentation for `participant-state` API [KVL-1094] (#10812)
60ffb79fb1 Command deduplication - Participant side/committer side command deduplication conformance tests split [KVL-1093] (#10784)
1e6540f8f2 Stop using `controller _ can` syntax in daml-lf tests (#10805)
0db15e6759 Ledger API test tool: ensure that test identifiers are unique (#10804)
f1e378a60b rotate release duty after 1.17.0-snapshot.20210907.7759.0.35a853fd (#10809)
1464177bf8 release 1.17.0-snapshot.20210907.7759.0.35a853fd (#10808)
deb8a8b740 [Divulgence pruning] Daml SDK docs update [DPP-536] (#10764)
8405ad58fd Enhance InMemoryLedger to use the ValueEnricher (#10757)
```
Changelog (since 1.17.0-snapshot.20210907.7759.0.35a853fd):
```
- [JSON API] Fixed a bug that prevented the JSON API to be aware of
packages uploaded directly via the Ledger API.
- The auth middleware will now consider that actAs credentials are valid
for readAs requests.
- [Ledger API Specification] Introduced a new conformance test about recort time in completions being monotonically increasing
ledger-api-test-tool - CommandDeduplicationIT tests only participant side deduplication, it should be disabled for KV ledgers. Added KVCommandDeduplicationIT which should be run for KV ledgers
- [Ledger API Specification] `Commands.deduplication_time` field has been deprecated, please use `Commands.deduplication_duration` instead.
ledger-api-test-tool - CommandDeduplicationIT tests only participant side deduplication, it should be disabled for KV ledgers. Added KVCommandDeduplicationIT which should be run for KV ledgers
[Ledger API Specification] Participant pruning of all divulged contracts is fully implemented: Participant operators can choose to prune all immediately and retroactively divulged contracts, by setting the newly-added prune_all_divulged_contracts flag in the ParticipantPruningService/Prune request.
```
changelog_begin
changelog_end
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@akrmn is in charge of this release.
Commit log:
```
35a853fd30 Async logging for http-json-perf tests and publish logs as a pipeline artifact (#10803)
a349217f78 get oracle perf test output (#10800)
5f448f2603 Cleanup DataSource initialization (#10794)
0e250d02e3 Fix parallel ingestion initialization (#10797)
b8bd5e6399 interface PoC: protobuf definitions (#10796)
a1da025b78 FreePort draw from outside ephemeral port range (#10774)
945a86c3f2 Fix a race in a test of the RecoveringIndexer (#10792)
4093bbd58c fix macOS Bazel cache (#10795)
ed99fe5aa5 ParticipantPruningIT added events for canton safe-pruning and multi-node synchronization (#10775)
5b28aefbce Deprecate kvutils rejections generated only by participant state v1 [KVL-1090] (#10791)
116d6a5994 DPP-577 Use BIGINT instead of TIMESTAMP (#10761)
cdf4bf1138 Check compatibility of timestamps (#10793)
098a08c58b Add Moisés to release rotation (#10789)
7ae199f027 Follow-up to h2 storage backend user/password in url: Review feedback switch to regex and move to pure function (#10541)
a5999abf8e remove dead code (#10765)
4b7391b0d5 LedgerApiTestTool prints skipped (excluded) tests (#10726)
a0f0fb3684 [DPP-418] Enable outstanding test case for private key decryption (#10778)
d750666f8e Do not drop details when converting between gRPC `Status` classes [KVL-1084] (#10745)
5ec9a234db Fix link to the gRPC health checks in the driver for PostgreSQL docs (#10779)
8538fe97b4 HA protected ledger initialization for append only schema [DPP-551] (#10705)
23b0fe7ac2 Fix exclusions for command service tests (#10777)
c4e0a755d4 LF: drop ad-hoc ImmArray builders (#10763)
303ba90595 participant-state: Re-enable integration test for command deduplication [KVL-1089] (#10751)
793253ca87 bump cleanup threshold (#10771)
4d67684737 Fix flaky DBLock tests (#10770)
751587125f Fix exclusions for command dedup in compat tests (#10767)
50fecfb9ef Wrap missing label names in quotes (#10749)
b1c6e87803 fix claims check in auth middleware (#10768)
5595d55c79 [JSON-API] Use the token from incoming requests to update the package list (#10602)
dbafea0e48 Add unit tests for DBLockStorageBackend [DPP-495] (#10746)
e5a6d70182 Added buffer size metrics for getTransactions/getTransactionTrees (#10744)
f76c868ee4 update NOTICES file (#10762)
bb908d04fb Create a link to party management service in main Ledger API page. (#10138)
e45d852ed4 Fixes participant to do retries on startup as waiting for DB connectivity (#10759)
f68a12930b Report some oracle_json_perf numbers on slack (#10754)
7270ee3c71 Handle dynamic port in auth middleware client (trigger service) (#10755)
ff1308ee3d [Docs] Add info on logs on Kubernetes & metrics in the ops section (#10525)
d26739087f Update buf image [kvl-1049] (#10752)
d2180cf60b Update exclusion for command deduplication to include full version range (#10750)
41881ba2ba Command dedup: migrate kvutils to use v2 services [KVL-1049] (#10679)
4525b8c265 [JSON-API] vanilla oracle_perf ci job (#10688)
16df8a5e35 [Short] Remove unused code (#10719)
b28afcf7ae [DPP-438] Update docs on metrics that no longer use <party_name> in their name (#10728)
0662025f4c Clarify what the `buf` image is and how it should be used (#10741)
9071a05c76 sandbox-classic: Reintroduce SqlLedger tests for the mutable index. (#10722)
f576cdfd06 fix flaky test in RecoveringIndexer (#9619)
f6a75b42f3 ledger-api-common: Do not mock final classes. (#10733)
e9c8af5024 Bump ghc-lib to include dropped parsing code for generic templates (#10735)
862a2901fb rotate release duty after 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10731)
12cb92b431 update compat versions for 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10737)
73c0de8db4 [DPP-578] Temporarily disable flaky test - Mockito plugin issue (#10734)
963bcb17eb release 1.17.0-snapshot.20210831.7702.0.f058c2f1 (#10730)
c11323ddb1 LF: Refactor engine test reinterpret (#10724)
a995fa3cd2 [DPP-574] Update docs about TLS: encrypted private key (#10727)
66970b7226 Not sharing the absolute Deadline object. (#10713)
c6c304b778 Improve script error on invalid script identifier format (#10702)
b748fd6b67 Support TLS in Daml Helper --json-api requests (#10709)
```
Changelog:
```
kvutils - protobuf rejections generated by the participant state v1 API are deprecated (Inconsistent, Disputed, ResourcesExhausted, PartyNotKnownOnLedger)
[Docs] Fixed link to the gRPC health checks in the driver for PostgreSQL docs
participant-state: Emit completions (CommandRejected) for duplicate commands when using pre-execution
- Fix a bug in the auth middleware where insufficient credentials could
still give access to list of running triggers.
- [JSON API] The cli option `--access-token-file` is now deprecated. It
is not needed anymore and you can safely remove it. Reason is that
the operations which prior required a token at startup are now done
on demand using the token of the incoming request.
- [Docs] Information was added in the `Operating Daml` section on how to aggregate logs on Kubernetes in conjuction with Daml services & what options exists for exporting metrics from daml services (not Kubernetes specific)
participant-state:
- Migrate to use only v2 Read/Write Services. This includes the use of new models for rejections/updates/submission results.
- Calculate deduplicate until for committer side deduplication based on the submitter given deduplication period
- if using the deprecated deduplicateUntil then just set the given timestamp
- if using duration then calculate the new deduplicateUntil by using the formula (submissionTime + deduplicationDuration + minSkew)
- if using offset deduplication then calculate deduplicateUntil by using the formula (submissionTime + maxDeduplicationDuration + minSkew)
- if the deduplication period is not set then we don't set deduplicateUntil
- Emit completions with status `ALREADY_EXISTS` for duplicate commands
- [Java Bindings] DamlLedgerClient.Builder allows to set a timeout for command using `withTimeout`.
- [Daml Assistant] The `daml ledger` commands now accepts `--tls` in
combination with `--json-api` to access a JSON API behind a TLS
reverse proxy.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@aherrmann-da is in charge of this release.
Commit log:
```
f058c2f16f DPP-368 clean up flags (#10711)
90ad24fd6a [Divulgence pruning] Prune immediate divulgence [DPP-513] (#10691)
183934b89b Command dedup: add columns to completions in append-only schemas [KVL-1057] (#10652)
27c13334b6 LF: Drop outdated TODOs (#10725)
9be577a7fb Enable pruning in the sandbox-classic when the append-only schema is used [DPP-567] (#10708)
9f072aeba0 Ledger-API Conformance test for Contract ID V0 (#10717)
867547c579 DPP-368 enable append-only flag in sandbox (#10710)
bdc511eaa9 [DPP-438] Change open-ended metric names into static ones (by removing partyName part) (#10706)
0c820063f1 [Divulgence pruning] Prune all divulged contracts only after migration offset [DPP-483] (#10661)
2555dbb30b Use soft references for values in the caches (#10715)
7fd590694e Add LedgerClientWithoutLedgerId next to the LedgerClient (#10681)
856c69c425 participant-integration-api: Increase a test timeout. (#10721)
b86d07d498 remove non functioning oracle json array indices (#10720)
f5e17567f9 sandbox-classic: Remove default parameters in `SqlLedger.Owner`. (#10718)
9ef3377864 LF: Update specification with Contract ID Comparability check (#10703)
e5c4734780 enable JSON search index on Oracle by default (#10539)
1ded42f185 [DPP-418] Protect TLS keys - follow-up cleanup (#10696)
3fcd986f0b Introduce a new `Offset` format [KVL-1063] (#10668)
a5781a6682 update NOTICES file (#10714)
89855054b5 participant-integration-api: Use deadlines, not deduplication times, for expiring trackers. [KVL-1009] (#10704)
65025c26b8 sandbox-classic: Add ErrorInfo metadata for rejections. [KVL-1048] (#10707)
97bda3ca36 LF: V1 Contract ID check in Preprocessor (#10687)
c2f90efbba Add CLI option to force disabling of participant deduplication (#10698)
6016633bb7 Construct ParsedModule directly in Daml Repl (#10701)
bbdf16aacf DPP-368 unhide append-only CLI flags (#10697)
a41b1349df Use the tracker retention period as the maximum expiry time. [KVL-1009] (#10700)
e750ba5c45 Make warning less scary. (#10699)
5f120bdf87 rotate release duty after 1.17.0-snapshot.20210824.7647.0.640fb683 (#10660)
a17253fb63 DPP-535 Verify postgres version (#10577)
301ce53a71 participant-integration-api: Add tests for ApiCommandService. [KVL-1009] (#10689)
bd01a211f4 [DPP-418] Protect Participant TLS keys (#10629)
7ee13243e1 update NOTICES file (#10695)
7c392f32dc update NOTICES file (#10693)
3db654e9dd update NOTICES file (#10690)
eff09c010a ledger-api-client: Wrap command submissions in a new class. [KVL-1009] (#10683)
d54adb2543 Ledger-API: Conformance tests for contract IDs suffixing (#10654)
aa2e869ede [Divulgence pruning] Pass divulged contract arguments through kvutils Write/ReadService [DPP-535] (#10598)
1a7831390e Disable DeeplyNestedValueIT suite against canton in Daml repo (#10686)
b5f9be3fbc participant-integration-api: Standardize tracker retention naming. (#10682)
2aa632e3aa ledger-on-sql: Do not increment the dispatcher head on start. (#10684)
eabb19d7f7 [ledger-api] Add deduplication_duration to deduplication period [kvl-1047] (#10676)
96ad9b5ab8 [Divulgence pruning] All divulgence events pruning [DPP-483] (#10634)
0b7980d6eb Update rules_haskell (#10674)
284edfc4cd Fix FlywayMigrations datasource (#10666)
adbe65fae1 Document ActionFail vs CanAbort (#10657)
52e7a6d928 update compat versions for 1.17.0-snapshot.20210824.7647.0.640fb683 (#10667)
f42e6b6877 Expose pending contracts in triggers (#10672)
7cc698948c Add multiple ways of specifying deduplication [KVL-1047] (#10601)
53be19f86c participant-integration-api: Ensure that all waiting, failed, and closed trackers are cleaned up. (#10662)
b27cde6440 participant-integration-api: Move tracker code around, and tidy up tests. (#10663)
fc9d35935a Drop alternative rules from dlint config (#10646)
5204d3ce7a Include committers in PartialTransaction root context (#10665)
387c68b9eb Normalize transaction values within the engine (#10648)
430c1cccf3 release 1.17.0-snapshot.20210824.7647.0.640fb683 (#10659)
ef239fdd1a participant-integration-api: Move `TrackerMap` code around. [KVL-1009] (#10653)
```
Changelog:
```
- [Sandbox] - Participant pruning is enabled in the sandbox-classic when the append-only schema is used
- [Ledger Client Scala Bindings] A new variant of the LedgerClient class
was added called `LedgerClientWithoutLedgerId`. This class does not
need a ledger id at initialization. It was added to allow skipping
any checks at initialization for use cases where either the
ledger id is not known at initalization or no valid token can be fed
at initialization for checking the ledger id. Furthermore for each
classes `ActiveContractSetClient`, `CommandClient`, `PackageClient`,
`TransactionClient`, `VersionClient` now exists a variant which
doesn't depend on a ledger id at initialization and instead requires
one for every function as parameter. Moreover the existing classes
are extending these classes with overriding the methods and setting
the default of the parameter with the given ledger id from
initialization. The class `LedgerClientWithoutLedgerId` already
makes usage of these variants e.g. `PackageClientWithoutLedgerId`.
- [Ledger Client Scala Bindings] The function `transactionSource` of the
class `LedgerClientBinding` now optionally accepts a token which is
passed on to the unterlying call.
- [JSON API] The Oracle database schema has changed; if using
``--query-store-jdbc-config``, you must rebuild the database by adding
``,start-mode=create-only``. See #10539.
- [Trigger Service] ``--help`` no longer advertises unsupported JDBC
options from JSON API.
- [JSON API] [EE only] By default, on Oracle, sets up a JSON search
index to speed up the queries endpoints. However, Oracle versions
prior to 19.12 have an unrecoverably buggy implementation of this
index; in addition, the current implementation fails on queries with
strings >256 bytes, with no way to disable the index for that query.
Pass the ``disableContractPayloadIndexing=true`` option as part of
``--query-store-jdbc-config`` to disable this index when creating the
schema.
See `issue #10539 <https://github.com/digital-asset/daml/pull/10539>`__.
- [Integration Kit] Changes the Offset format to contain a version and therefore reduces the highest index size by one byte
- [Ledger API Server] The command deduplication time is no longer used
for determining the period of time to track the command before giving
up. Instead, the gRPC deadline is used. If no deadline is provided
(or if the deadline exceeds the command tracker retention period), the
tracker retention period is used instead.
- [Daml Repl] Fix a bug where bindings with out of scope types would result in error in following lines.
- [Sandbox, participant] Added a flag to enable a new append-only database schema.
This schema was designed to support significantly higher performance.
In a future release, all applications will automatically migrate to the new schema.
- [Ledger API Server] The command service now uses the tracker retention
period (typically specified with the ``--tracker-retention-period``
command-line argument) as the maximum time to wait for a command to
arrive on the completion stream. After this time, the command will
time out, though it may still complete in the future. Previously, the
deduplication period was used, but it was likely the tracker would be
terminated before that anyway.
The default tracker retention period is 5 minutes, unless otherwise
specified.
- [DPP-418] [Participant] Add support for supplying server's private key as an encrypted file and then decrypting it with the help of a secrets server.
[Integration Kit] KV-based ledgers pass contract instances through the Write/ReadService, removing the need
for backfilling divulged contract lookups.
Note: KV Ledgers that have been created before this change will still be relying on backfilling lookups of divulged contracts,
hence pruning of all divulged contracts may result in failing lookups for divulged contracts.
ledger-api - add `deduplication_duration` as a future replacement for `deduplication_time` in the command proto definition
ledger-api - Command deduplication period can now be specified by setting `deduplication_offset` instead of `deduplication_time` (only valid for v2 WriteService). This change is backwards compatible.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@SamirTalwar-DA is in charge of this release.
Commit log:
```
640fb68342 Make Index DB enable multiple party additions [DPP-546] (#10623)
b22de6893b LF: Contract ID suffix check in Preprocessor (#10642)
7b94b0674e Map shortened scala test suite names to long names on Windows (#10628)
6e4a24c28c participant-state: Generate correct gRPC error codes by v2 `WriteService` [KVL-1081] (#10656)
663781a159 Update curl 7.73.0 --> 7.78.0 (#10655)
a471f15aa8 Dpp-558 Fix startexclusive queries on oracle (#10649)
e99254fd3e Augment `completion.proto` with deduplication-related info [KVL-1057] (#10619)
a00608c8b3 participant-integration-api: Accommodate changes to max dedup time. (#10650)
29c546c949 [Divulgence pruning] Added `prune_all_divulged_contracts` to PruneRequest [DPP-534] (#10635)
dea57ca099 In-memory fan-out optimizations (#10558)
77eb366627 [JSON-API] key_hash field to speed up fetchByKey queries (#10631)
5001329c9d LF: Comparisons fail at runtime if comparing local vs global CIDs (#10630)
055be4b3cb Disable deprecation warnings for data-dependencies (#10647)
c1559357f6 clean-up gsg README (#10641)
8501832757 DPP-468 StorageBackend tests (#10529)
4e08b47dd1 update NOTICES file (#10645)
733590d591 ledger-api-health: Use the Scala health status values everywhere. (#10640)
5b837ec2d9 Ledger API: add `buf` checks [KVL-1045] (#10625)
f77cd0a81c participant-integration-api: Attempt to fix RecoveringIndexerSpec. (#10639)
9d7f60f03e participant-integration-api: Fix a flaky test. (#10637)
4a9331ca55 Upgrade Nixpkgs [KVL-1045] (#10624)
01b6e89551 update compat versions for 1.17.0-snapshot.20210817.7604.0.0c187853 (#10610)
b578b0e7cf Reminder to put an empty line between subject and body (#10638)
c0fbad1f6e participant-integration-api: Remove `limitMaxCommandsInFlight`. (#10626)
b4af6d1155 Canton testing: Mark one more DeeplyNestedValueIT test flaky (#10636)
e807f4a918 Upgrade to a newer canton version (post 0.27.0 snapshot version) (#10632)
c4513f2eea Oracle append-only schema: enable contract id index on participant_events_xxxx tables (#10633)
3598e0928c LF: Drop contract ID Freshness check (#10620)
37c999e237 ledger-on-sql: Increase the concurrency for conformance tests. (#10622)
46e8c7d762 DPP-460 Extract constant for event sequential IDs (#10564)
121047e0ea DPP-460 Parameter storage consolidation (#10472)
569612ab73 Drop broken symlink config (#10616)
6d0109fca8 Support $ in daml-lf identifiers in the parser (#10609)
c38703ec84 participant-integration-api: Store a status gRPC protobuf. [KVL-1005] (#10600)
0af5b49484 make FinalReason a case class (#10614)
8dd136fc7d bazel-tools: Replace `runner` with either `runner_with_port_check` or `runner_with_port_file`. (#10615)
19c3d28ebf Remove GenMissingString class because it is not used (#10608)
3227e860e0 Use the port file and dynamic port generation in client/server tests. (#10604)
fb19bcb7f3 fix gsg-trigger template (#10611)
b2077026d6 rotate release duty after 1.17.0-snapshot.20210817.7604.0.0c187853 (#10606)
975a5fb32b Move DeduplicationPeriod to ledger-api-domain [KVL-1047] (#10590)
f8a1820cc8 release 1.17.0-snapshot.20210817.7604.0.0c187853 (#10605)
64abf8aaa9 update NOTICES file (#10607)
```
Changelog:
```
[Integration Kit] Corrected gRPC error codes returned by v2 `WriteService` adaptor.
- [Ledger API Server] The API server manages a single command tracker
per (application ID × submitters) pair. This tracker would read the
current ledger configuration's maximum deduplication time on creation,
but never updated it, leading to trackers that might inadvertently
reject a submission when it should have been accepted. The tracker now
reads the latest ledger configuration.
- Update schema version for http-json-api query store with new key_hash field
- Improved performance for fetchByKey query which now uses key_hash field
participant-state - move `DeduplicationPeriod` to ledger-api-domain
dc4629fff9 update NOTICES file (#10580)
8b0a0e7bbc update NOTICES file (#10578)
4b8b67a1b5 Upgrade Scalatest to v3.2.9. (#10576)
41e60f7302 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898cc9 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90dffd Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc6d6 rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44fbd update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa7c0 Bump bazel max jvm memory (#10569)
1cc136c033 update NOTICES file (#10565)
c69880ccf1 ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f80c Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f36b6 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
Commit log:
```
0c1878530b test: test for platform independent dars (#10535)
3822a8c886 ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de564f Introduce dependency on Oracle image version (#10597)
f8c0a35940 rewrite trigger docs to follow gsg (#10509)
aec601f929 Fix badly versioned empty tx (#10596)
d92440471c daml ledger export: export all parties (#10588)
fb09b72f96 Improve divulgence warning message. (#10595)
fd9d8724fc Upgrade to Oracle 19.12 (#10589)
386965c466 Upgrade Flyway to v7. (#10594)
654d2eee1b Bump url-parse to address dependabot alerts (#10593)
01b8e3011f Simplify loading of logback file (#10592)
19bfdbee00 update NOTICES file (#10591)
5f7a3691a5 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f32d4 Upgrade path-parse to 1.0.7 (#10587)
5c8025269d ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccfc4e Normalize transactions & values as a separate pass (#10524)
99e1d78bb8 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc39cb Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2b96 Skip subject in changelog check (#10584)
244262dfac Handle imports for builtin types correctly in Daml export (#10575)
1fc281f07e ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15465 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb933 JSON API: log ledger connection errors at every attempt (#10581)
933b58e3aa Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef894447f Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52cdc changelog_begin (#10579)
dc4629fff9 update NOTICES file (#10580)
8b0a0e7bbc update NOTICES file (#10578)
4b8b67a1b5 Upgrade Scalatest to v3.2.9. (#10576)
41e60f7302 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898cc9 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90dffd Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc6d6 rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44fbd update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa7c0 Bump bazel max jvm memory (#10569)
1cc136c033 update NOTICES file (#10565)
c69880ccf1 ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f80c Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f36b6 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@remyhaemmerle-da is in charge of this release.
Commit log:
```
b5e9d86114 Moved DamlSubmission* into separate proto file [KVL-980] (#10362)
0da814d250 Let stable packages bypass LF version restrictions. (#10377)
c97cbcabfd [JSON-API] Validate schema version & add minimal options for schema creation (#10374)
ebb8fab97e Add a ContractDao benchmark for payload queries (#10426)
85af078085 LF: parser for LF versions (#10424)
b976c9cf4c Disable autocommit for hikariCP for http-json-api db conn (#10427)
3ca46a4bcb Removed unused import. (#10425)
4d1249390e Introduce buf checks [KVL-980] (#10411)
1c4ae5023d Revert "Upgrade hikari to latest jdk8 version (#10406)" (#10421)
fe1b642e02 Don't fail if logs directory exists already (#10423)
7c88b56e16 participant-integration-api: Fix completion debug log output. (#10415)
fc305e6c2e [JSON-API] Shutdown on startup if the db connection is invalid (#10360)
6db58695ad Update vcredist (#10417)
4b55f1ab6a Connection pool for Contract Dao (#10359)
72cf2f36d3 LF: replace bazel keyword stable by default (#10410)
2bdcb7b69f update NOTICES file (#10414)
39c6e0b329 Fix oracle message too long error (#10413)
2094e24f28 Indexer ValidateAndWaitOnly startup mode for canton participant HA (#10290)
ad13a86947 Windows dev-env Powershell 7 compatibility (#10408)
3cedd83447 Easy to parse ledger-api-bench-tool logs (#10320)
16ff20c04f Fix links in changelog (#10409)
b325e8a8b8 participant-state: Remove `WriteService#rejectSubmission`. (#10407)
9be520c3e8 Keep the participant-state API prefixed. [KVL-1002] (#10405)
d88dc7129b Upgrade hikari to latest jdk8 version (#10406)
8b337bda4b Publish ledger-indexer-benchmark (#10401)
9e05f38f51 ledger-api-domain: Store the deduplication duration in `Commands`. [KVL-1002] (#10403)
9c064dac35 Allow imports of internal modules (#10397)
b9518ce2f4 participant-state-metrics: Add wrappers for v2. (#10404)
c3a3d60083 don't call Gary, he's on holiday (#10400)
221d0a0689 use doobie 0.9.0 Fragment-in-Fragment interpolation in json-api db-backend (#10399)
17709b5ba3 use a single SQL query for any number of json-api query pairs (#10344)
6a166841c5 Stop publishing the db-backend artifact (#10396)
1bed05fec2 Treat KeyWithMaintainers field structurally in all node types. (#10392)
d7077e154f Introduce locally-defined `Rejection` reasons instead of the participant-state rejection reason type. [KVL-1002] (#10376)
96f048330a [Divulgence pruning] Conformance tests implementation [DPP-484] (#10385)
28c5e9a435 update NOTICES file (#10386)
387945298d fix cut&paste typo bug; there are no insig lookup tweaks! (#10389)
7df9758c2f Daml export: make paths relative to daml.yaml (#10388)
90c3582ee8 treat exerciseResult structurally for isReplayedBy (#10381)
27a0c694bc Stop swallowing error message in non-repudiation conformance tests (#10387)
22b6101bc8 rotate release duty after 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10346)
5242e2c618 LF: drop old serializability check for Values (#10382)
ee755306fe LF: Specify nesting constraint for serialized values. (#10375)
4a33c03f72 LF: Add check of nesting in SValue.toValue (#10370)
91529ee43d clear up record specifications in http-json integration tests (#10366)
e8247afc6e update compat versions for 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10354)
74751ba9a8 Populate workflow-id in the test-tool scenarios (#10372)
da9f8e3007 clean-up perf tests (#10355)
42b70ad31d Fetch actingParties always non-empty for supported versions (>=1.6) (#10357)
37ff1a60fe ledger-configuration: Return a structured error from checkTime. [KVL-1002] (#10373)
66284c17f2 Limit length of package ids to 64 characters (#10368)
a56cfead28 even earlier mount failure detection (#10371)
1bc0ccdac7 update NOTICES file (#10367)
9c9b91eca7 Support deletion of a large number of contracts (#10353)
1b5f99e051 Stop printing stacktrace on logging setup failures (#10364)
c0a24fe9b6 HA PoC behind a feature flag [DPP-426] (#10227)
63739fa712 Add conformance test for deeply nested values (#10319)
faf479e8c8 LF: add context in LookupError (#10314)
68dcda0f29 Drop unused textType from JSON API Oracle queries (#10356)
49745f6e47 Re-add `application_id` to `DamlCommandDedupKey` [KVL-1000] (#10341)
27d439d546 LF: Compile netsed ELets using constant stack (#10337)
628aa22f51 kvutils: Refined transaction validation [KVL-1015] (#10066)
d1e84c8563 participant-integration-api: Fill out stubs in ApiSubmissionServiceSpec. (#10349)
905d8add44 remove duplicated rule in create-daml-app dlint (#10352)
bdc3e509b3 Separate ledger export related message from other kvutils messages [KVL-980] (#10343)
70e8ff46da participant-integration-api: De-spacify PostCommitValidationSpec. (#10350)
186d279c81 release 1.16.0-snapshot.20210720.7404.0.b7cf42d1 (#10345)
e58c7ba3b4 fix template dot-files (#10342)
cf2b61e6ac participant-state: Remove the aliases to `Ref.LedgerString`. (#10325)
8148137259 [JSON-API] Include the logging context in the ledger client for consistent logging (#10332)
c9666c7fe0 Drop unnecessary mutability from speedy OnLedger state (#10340)
60dd96a3eb update NOTICES file (#10347)
```
Changelog:
```
[Integration Kit] Moved definitions of `DamlSubmission` and `DamlSubmissionBatch` to a separate proto file under the package `com.daml.ledger.participant.state.kvutils.wire`. In case you are directly referencing these messages you will have to update your imports.
- [JSON-API] Schema versioning was introduced to the db schema. Because of this the field `createSchema` in the jdbcConfig was deprecated. Via the field `start-mode` you can specify:
1. `create-only`: This is equal to the behaviour of `createSchema=true` so the db schema is created and then the application terminates.
2. `start-only`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the application terminates. This is also the replacement of `createsSchema=false`.
3. `create-if-needed-and-start`: With this the schema version is checked and if no version or an version was found which is not equal to the internal schema version then the schema will be created/updated and the application proceeds with the startup.
4. `create-and-start`: Similar to the first option but instead of terminating the application proceeds with the startup.
- [JSON-API] The json api now correctly shutdowns at startup if the provided db connection is invalid in case of `createSchema=false`
simplify oracle migration scripts
- [Daml Compiler] Imports of internal modules from stable packages are
no longer illegal. Previously, the compiler raised an error when it
encountered imports of internal modules such as
`DA.Internal.Template`. Such imports are now accepted by the compiler.
Note, however, that internal modules are still not part of the stable
API. Fixes https://github.com/digital-asset/daml/issues/10379
[Integration kit] Extended the Ledger API test tool with tests for the pruning of all divulgence events.
- [Daml export] The generated paths to data-dependencies DALFs are now
relative to the generated daml.yaml. Fixes
https://github.com/digital-asset/daml/issues/10378.
- [JSON API] Fix an error where transactions that delete a large
number of contracts resulted in stackoverflows with the PostgreSQL
backend and database errors with Oracle.
[Integration Kit] The command de-duplication key now also includes the daml application ID
* [Integration Kit] Made `daml_kvutils.proto`'s location follow its proto package and moved `LedgerExportEntry` into a separate proto file. You may have to update your proto import statements in case you are directly importing proto files from the kvutils library.
- [JSON-API] Connection tries from the json api to the ledger now include the logging context, more specifically the instance_uuid is included in each logging statement.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@robin-da is in charge of this release.
Commit log:
```
b7cf42d173 Upgrade doobie to version 0.13.4 (#10326)
1a723149d4 fix broken triggers link (#10336)
271f9daedd Added back DeduplicationPeriod.DeduplicationOffset [KVL-998] (#10324)
b701caab7d Report divulgence warning at commit location. (#10329)
742bdcbeea Describe prometheus in daml-on-sql README (#10307)
93d0ed6821 [Mutable cache] Resolve with full lookup on negative cache read-through lookups [DPP-501] (#10262)
f59951b755 document min npm version (#10330)
d982e16e99 fix besu listing (#10328)
c3bc26fd1c daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_. [KVL-1002] (#10323)
cf2f79db4e Register divulgence warning as a diagnostic (#10289)
8360e9f75a Mark Java Bot tests flaky (#10322)
a6bf892f72 [In-memory fan-out] CLI parameters update (#10250)
5fa49c3312 Disables flaky test suite of InstrumentedSource.bufferedSource (#10321)
504c231732 Use a combined template id, stakeholder index (#10315)
9a01065f29 participant-state: Remove the Daml-LF transaction and value aliases. [KVL-1002] (#10317)
159728d716 LF: use Either by default in all archive reader API (#10295)
449a72a86f increase ES memory (#10318)
80b65f6def [DOCS] Add documentation for the JSON API metrics (#10312)
6abb903a7f [JSON-API] Put the request & response bodies into the log ctx if the log level is debug (for specific statements) (#10298)
3c0010b38a detect mount issue earlier (#10313)
0e09f3ba08 Remove optLocation field from transaction nodes. (#10301)
e62852fbfd Make ledger API test tool's 'max-connection-attempts' into an option (#10311)
bd12170af2 participant-state: Remove the `ParticipantId`, `PackageId`, and `Party` aliases. [KVL-1002] (#10308)
fed44970ae Extend JSON API ContractDao query bench’s with different tpids (#10309)
bd35f80efa ledger-configuration: Correct the protobuf package. [KVL-1002] (#10302)
d69328d281 Ledger API Test Tool: add a `--max-connection-attempts` command line option [KVL-977] (#10297)
a3b861eae8 refresh es cluster (#10300)
63332d0667 update NOTICES file (#10306)
1ee53c0736 participant-state-index: Use `Ref` directly. (#10299)
0eba812109 Remove trace_context field from Ledger API [KVL-1021] (#10256)
88886beb8d ledger-configuration: Add tests for the `Configuration` decoding methods. [KVL-1002] (#10293)
cb29f34d4b ledger-offset: Move `Offset` to a new package. [KVL-1002] (#10296)
b899ae1d37 Codeowners: Add @nicu-da to the kvutils code review. (#10292)
f033bc2bbb LF: Clean legacy from archive proto + TEXT_TO_CODE_POINTS typo (#10286)
70b12f04d0 ledger-on-sql: Remove conformance tests for H2 and SQLite files. (#10291)
a9a0b70b85 ledger-configuration: Extract the configuration from participant-state. [KVL-1002] (#10279)
d96b54eb23 Use Text not String in exception docs (#10287)
53822537e4 Refresh json API materialized view on statement (#10285)
6e447c567e Patch export filtering of GHC.Types.[] in damldocs (#10282)
d565cae913 DPP-457 Add UpdateToDbdto tests (#10116)
814442eb80 update NOTICES file (#10284)
0493480e44 rotate release duty after 1.15.0-snapshot.20210713.7343.0.1f35db17 (#10268)
84094e0b6b update compat versions for 1.15.0 (#10281)
ba8e829c8d [In-memory fan-out] Oracle support (#10263)
a8f190214c LF: change type from Try to Either in archive module (#10277)
c84b37a67f Release SDK 1.15 (#10269)
980d96b44a Fix status check in collect_build_data (#10278)
191d3b82ce Add unit test to SequentialWriteDao [DPP-456] (#10265)
2271f93b69 Fix crash on duplicate submissionId (#10242)
906184ce1a LF: Simplify UniversalDarReader (#10271)
42c487ba96 [JSON-API] Refactor Endpoints.scala to use path directives etc. (#10274)
684040159f release 1.15.0-snapshot.20210713.7343.0.1f35db17 (#10267)
0df70fa56a Document bazel-testlogs directory in BAZEL.md (#10276)
44204bb932 Add a benchmark for contract insertion in the JSON API (#10272)
6107f8aa64 Ignore failure to upload log failures (#10270)
caf85a2270 LF: rationalize archive Parser/Reader/Decoder (#10239)
0043b81f00 Add a ContractDao benchmark (#10259)
21a9e62f27 Add disclaimer to participant state v2 API [KVL-998] (#10254)
6c63f96ba9 participant-integration-api: Move `SeedService` here. (#10261)
58c0b4694a kvutils: Simplify usage of Rejections [KVL-1015] (#10258)
41aba1ca39 kvutils: Move unrelated test cases to a proper test class [KVL-1015] (#10257)
```
Changelog:
```
- [JSON-API] You can now find a section `Metrics` in the http-json api documentation explaining how to enable metrics and which are available
- [JSON-API] Logging of the request and response bodies are now available for appropriate requests if the chosen log level is equal or lower than DEBUG. These can then be found in the logging context of the request begin & end log messages (The field names in the ctx are "request_body" and "response_body").
- [Integration Kit] The *ledger_configuration.proto* Protobuf definition
has been repackaged under ``com.daml.ledger.configuration``, and the
Java and C# packages have been renamed accordingly. If you are using
this Protobuf definition, you will need to update your imports. The
Maven artifact name has not changed.
- [Integration Kit] Add a `--max-connection-attempts` command line option to the Ledger API Test Tool
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
- [Integration Kit] The ``Offset`` type has been moved to a new Maven
package, ``ledger-offset``, from the ``participant-state`` package.
The Java package has been renamed to ``com.daml.ledger.offset``. If
you are using this type, you will need to update your dependencies and
imports.
- [Integration Kit] The ledger configuration classes, ``Configuration``,
``LedgerInitialConditions``, and ``TimeModel``, have been moved from
*participant-state* to a separate package named
*ledger-configuration*, in the Java package
``com.daml.ledger.configuration``. You will need to update your
dependencies and imports.
- [Integration Kit] ``TimeModel`` has been renamed to
``LedgerTimeModel``. If you are using the ledger configuration classes
directly, you may need to update your code.
Fix crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participants.
Extend ledger-api-test-tool to cover the duplicate submissionId cases.
- [Integration Kit] The class ``SeedService`` has been moved from the
*participant-state* Maven package to the *participant-integration-api*
Maven package, under the Java package name
``com.daml.platform.apiserver`` to reflect its usage by the API
server, not the participant state API. If you use this class directly,
you will need to change your imports.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* release 1.15.0-snapshot.20210713.7343.0.1f35db17
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@akshayshirahatti-da is in charge of this release.
Commit log:
```
1f35db17c7 ledger-on-sql: Use random log entry ID allocation so we don't depend on `SeedService`. [KVL-1002] (#10255)
b8e2198873 Separate traces from warnings in engine. (#10253)
0a7f2b1a1b [JSON-API] Small refactoring to start using akka http routes internally (#10252)
6e8ec1d618 LF: Drop old depreated code (#10251)
de7a08fa7b [In-memory fan-out] Handle `null` submitters in getTransactionLogUpdates (#10248)
584169a2cc Participant state v1-to-v2 adaptors [KVL-1002] (#10246)
e4d7cd764a LF: Deprecate com.daml.lf.data.TryOps.Bracket (#10249)
b59f36591d [JSON-API] Correctly extract the request source URL/IP (#10244)
edaf541063 Adapt `ledger-api-bench-tool` to work with LR [DPP-424] (#10171)
6fe6ae075d LF: Make DarReader ZipEntries immulatble (#10243)
05e5218d06 Consolidating EventStorageBackend (#10224)
3fd3abfb26 Instrument buffer after contract state events stream (#10209)
643a2de006 v2 participant state API draft [KVL-998] (#10210)
5abcd0482e Adds Unit tests for parallel-ingestion [DPP-455] (#10238)
3cdedcf885 kvutils: Extract validators from TransactionCommitter [KVL-1015] (#10235)
00d622f268 Make @UNTIL-LF exclusive, add @UNTIL-LF-FEATURE. (#10240)
2bcbd4e177 es: switch to persistent nodes (#10236)
7f4bc2a472 update compat versions for 1.14.2 (#10233)
4eba4b00e8 Ledger: decouple BD value serialization library from archive library. (#10218)
729afa8c7b Write a few lines on how to run tests against an Oracle database (#10230)
2b67ebb5d4 tf: refactor appr var (#10232)
202b7f7ae7 add akshay to appr team (#10229)
2c5410c229 Release SDK 1.14.2 (#10226)
89d87ad17c update compat versions for 1.14.2-snapshot.20210708.7135.0.aa297840 (#10223)
48393444df TODO Cleanup: QueryNonPruned [DPP-459] (#10225)
0bea5e3ef7 Allow retriable lookupMaximumLedgerTime on contention (#10211)
ca2a9e9bbb Release 1.14.2 snapshot (#10222)
999577a1a7 tweak ES cluster (#10219)
6680d368a1 Warn on DA.Exception import (#10201)
f19f5b0821 LF: Simplify DarReader (#10217)
8f1a8f2390 Release DAML SDK 1.14.1 (#10204)
ebb76dca4c LF: reorganize errors in com.daml.lf.archive (#10213)
69646451f2 Improvements for validation tests (#10214)
5f124c3b64 Avoid collision in execution log postfix (#10205)
9e27ae0d85 Reshuffle cocreature in daml-lf codeowners (#10215)
41b8448b17 LF: Simplify archive reader. (#10208)
38734f02d7 es-feed: ignore invalid files (#10207)
6586dde11e Move exercise context computation to the client side (#10199)
c92c67832f Mark Daml profiler as stable (#10203)
cb1f4ec773 ci/windows: disable spool (#10200)
2326d425bc Publish execution logs on unix platforms (#10194)
42693452ce Drop cleanup of node_modules (#10195)
c8faed8a17 Deduplicate Java codegen running instructions (#10185)
e4c8c390f2 update compat versions for 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10196)
0d881f5e2b Improvements to the documentation with regards to offsets (#10180)
1d5ba4fa42 feed elasticsearch cluster (#10193)
e2bdca6be9 Use PartialFunction for more concise code. (#10191)
fe98c48b65 release 1.14.1-snapshot.20210706.7133.0.8ec16e94 (#10186)
4fe626a055 Drop logging on CompletionSource (#10190)
582aa5f08c Fix a typo in an exceptions example. (#10188)
8750c0c47c reduce noise in Scala bindings (#10187)
98b5ffee01 Add divulgence warning and test in script service. (#10179)
8578e56aa7 Tests for transaction validation (#10167)
05a72a3a15 update compat versions for 1.15.0-snapshot.20210705.7286.0.62aabcc4 (#10184)
7b93923c16 [In-memory fan-out] Performance optimizations [DPP-470] (#10127)
6c49619565 Document how to run the Java codegen from the Daml assistant (#10181)
61aa774988 Release SDK 1.15 RC (#10182)
```
Changelog:
```
- [JSON-API] If the service is put behind a proxy filling either of these headers X-Forwarded-For & X-Real-Ip then these will now be respected for logging the request source ip/url
- [Daml Compiler] Importing DA.Exception on LF < 1.14 now produces a
warning that exceptions require Daml-LF >= 1.14.
- [Daml Profiler] The Daml profiler is now a stable feature.
[Docs] Improvements to the documentation with regards to offsets
[Docs] Document how to use the Java codegen using the Daml assistant
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.16 not 1.15
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
Commit log:
```
84e1f3a7f1 participant-integration-api: Move transaction requests to structured logging. [KVL-996] (#10141)
b79d02e28e `ledger-api-bench-tool` - exposed metrics for Prometheus [DPP-471] (#10103)
b61e519a3a Release ledger API test tool per LF version (#10142)
2dfe026cc2 add ES cluster (#10144)
ef9a04caf5 Divulgence crash tests [DPP-433] (#9942)
bb4641756f Daml assistant capitalization (#10140)
a6ee10be9a Move visibility checks into speedy. (#10136)
c764fbe573 contextualized-logging: Introduce different logging types for more structure. [KVL-996] (#10134)
678cab0327 10050 append only schema on oracle (#10051)
f5e5062470 ensure that signatories and observers are disjoint in ledger-api CreatedEvents (#10123)
ffc88d52b4 print version of the ledger api test tool as part of the report (#10119)
621cfa4a91 Resolve contract keys conflicts in disclosed contracts (#9948) (#10034)
f1ffd52e08 Java bindings: add DamlRecord, deprecate Record (#10132)
3df2566305 Scenario: move Scenario Error Throwable to scenario package (#10075)
7521ec9988 [JSON-API] Include contract id in the logging ctx when logging exercise commands (#10131)
e0e333377c update NOTICES file (#10129)
398300b76f LF: Move Speedy Interpretation Error to transaction package (#10091)
d4150ac078 Refactor error reporting in Daml Repl (#10118)
0108fedd4e update NOTICES file (#10112)
6af36fe1e3 Fix Tx.Metadata for normalized transactions (#10108)
7745693648 Remove duplicate index (#10084)
1b617ae22d DPP-428 Add missing indices for index initialization (#10083)
af9382ce92 contextualized-logging: Reduce the API surface and avoid name collisions. (#10102)
01d6770450 json-api perf tests: combine large-ACS frequencies combinatorially instead of in lockstep (#10101)
a44afcff42 Upgrade nixpkgs (#9908)
9498d1509b [DOCS] Let jwt.io link to an already filled in working payload (#10026)
4affb053e9 Force newer version of glob-parent in Navigator (#10105)
f745f10394 LF: Realease LF 1.14 (#10077)
05056ddd8c Cut a flaky expectation some more slack (#10106)
46a66e2f2a Upgrade jest to the latest version (#10107)
84e329885f update compat versions for 1.11.2 (#10104)
a6b536f3c4 Compiler: Make LF 1.13 the default output (#9907)
29ddc88dcb remove redundant spaces in the log format (#10093)
01e329f2de DPP-432 Add exception tests to the JdbcLedgerDao suite (#10040)
3d79cbf2b1 Bump compat versions (#10096)
5bf7d8f570 Fix speedy perf reporting (#10100)
0970821ce7 Bump cpu alloc for build-and-lint (#10099)
cbca779657 [In-memory fan-out] Ensure getTransactionLogUpdates with max fetch size (#10064)
c756153951 Release SDK 1.11.2 (#10094)
3a4235400d rotate release duty after 1.15.0-snapshot.20210622.7213.0.d867d904 (#10086)
ecc2d11536 recommend some daml compiler warnings to enable (#10082)
b2bb45e4d9 Bump daml repl timeout (#10089)
db60d15b5e Log ledger api validation failures at info level (#10080)
f0dc025ac9 Release 1.15 snapshot (#10090)
```
Changelog:
```
- [Ledger API Server] The amount of data logged in the API transaction
service has been reduced at INFO level. Please enable TRACE logging to
log the request data structures.
- [Integration Kit] - ledger-api-bench-tool - exposed metrics via Dropwizard metrics (e.g. for Prometheus)
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
In case a contract key is already present in a past contract in the contract table nullify it. This gets rid of contract keys of previously disclosed contracts. We never discover that disclosed contracts get archived, so we get conflicts on such past keys
[Java bindings] In order to avoid clashing with `java.lang.Record` (introduced
in Java 14), `com.daml.ledger.javaapi.data.Record` has been renamed to
`com.daml.ledger.javaapi.data.DamlRecord`. The old name has been used to
denote a sub-type of the newly renamed one, so it can still be used, but it has
been marked as deprecated.
[Java codegen] The Java codegen now uses the `DamlRecord` type wherever `Record`
was used before.
- [JSON-API] The contract id is now also included in the logging ctx when logging exercise commands
- [LF] Add support for Excepction
- [Compiler] Default ouput LF version is now 1.13
Log ledger api validation failures at info level
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Commit log:
```
adeba20659 DPP-403 log flyway progress (#9964)
401069ef00 prepare for upgrade to Scalaz 7.3.3 (#9997)
3ad9cd58d9 [JSON-API] Log templateId & choice name (if present) on command submissions in the json API (#10005)
af144e8592 LF: Structure package loading errors (#9980)
3583fd1885 [JSON-API] Log source and path of incoming http requests & the response with info level (#10007)
e49c7c8db7 [In-memory fan-out] BufferedTransactionsReader implementation [DPP-413] [DPP-414] (#9882)
acb77336c8 Release 1.14 RC 2 (#10006)
3d2d4b9696 Don't duplicate log failed futures in the http json api (commandservice specific ones) (#9990)
0c32cf2968 Clarify the semantics of DIV_INT64 in LF spec. (#10003)
b79e739196 Fix Type (#10004)
998f106ee6 Drop pkgIds argument from Engine.validatePackages (#9986)
ae5010f3ef [Mutable state cache] Start cache from latest ledger head (#9998)
dd42a5a641 update maven_install_2.12.json (#9999)
66d38cd24d Storage query abstraction first sketch (#9959)
c75013622c Just print out the operator name for the operation logging in CommandService.scala (#9989)
b6faace335 Changing oracle backend from using varray to json array (#9943)
603a4a4b42 Add date to log statements in the HTTP JSON service (#9988)
836a82a6b6 Add proper error handling for missing contract keys (#9972)
85667af7be Address npm vulnerabilities (#9978)
d7d6e37785 LF: Structure Engine Errors. (#9971)
0adbbdcd58 update Daml Hub auth in create-daml-app (#9973)
997a7d449c Use symlinks on Windows (#9969)
e4258e61b1 docs: tweak Exception intro (#9967)
b2c4ef813d rotate release duty after 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9933)
de7a462082 Log call to submitAndWaitRequest with the command id provided in the log ctx (#9940)
630d021331 Collect build-event-stream (#9831)
26e199a719 Rerelease 1.11.2 snapshot (#9965)
8399039a8a Remove broken dep from release job (#9963)
e1db529162 DPP-390 configurable event decoding parallelism (#9931)
cc7af93afc [docs] Remove link to the #daml-contributors (#9928)
f7ac3ba784 Release 1.11.2 snapshot attempt 2 (#9962)
4634147ed1 Adds Indexer state to GRPC health checks [DPP-434] (#9951)
28c567b53d Release SDK 1.11.2 (#9957)
5f0f5756bf Define IncompleteTransaction for use by scenario-service (#9952)
b7fd8338c4 Increase `eventually` timeout when waiting for queue to be killed (#9955)
ca0fb16718 Fix collect_build_data job (#9956)
42030b25ed Skip Scala_2_12 job on releases (#9954)
5e1ddfbd5e LF: Fix specification of ArithmeticError (#9949)
3093a98b87 Forward port additional contract keys validation tests (#9944)
e541558f18 update NOTICES file (#9946)
db792a9774 Proxy ledger API health endpoint in JSON API health endpoint (#9945)
13767b5c5d Load the correct logback file for the http json service respecting the deployment situation (#9938)
4e49cf6814 `ledger-api-bench-tool` - Active contracts stream, completions stream [DPP-398, DPP-399] (#9857)
fce8e54469 release 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9932)
ab55fb2546 Handle module-prefixes in Java codegen (#9929)
7375e28271 Force resolution for css what to 5.0.1 (#9925)
```
Changelog:
```
- [JSON API] The template id & choice name (if present) are now logged on command submissions in the Json API (at trace level)
- [JSON-API] The source and the path for incoming http requests are now logged
- [JSON-API] The http response for a request is now logged
- [JSON API] Errors which arise from the CommandService are not logged twice anymore (thus reducing noise)
- [JSON API] log statements now include the date next to the time
- [HTTP-JSON] Calls which trigger a submitAndWaitRequest are logged with the command id provided in the log ctx
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
- [JSON API] The healthcheck endpoint on the JSON API now proxies the
health check endpoint on the underlying ledger. Previously it only
queried for ledger end to determine ledger health.
[HTTP-JSON]
- fixed that json log output could not be enabled via cli options except via usage of env vars
- [Integration Kit] - ledger-api-bench-tool - reading active contract streams
- [Integration Kit] - ledger-api-bench-tool - Reading completions stream
- [Java Codegen] The Java codegen will now pick up the
`module-prefixes` field from `daml.yaml` which can be used to handle
module name collisions between different DALFs.
```
CHANGELOG_BEGIN
CHANGELOG_END
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
Commit log:
```
adeba20659 DPP-403 log flyway progress (#9964)
401069ef00 prepare for upgrade to Scalaz 7.3.3 (#9997)
3ad9cd58d9 [JSON-API] Log templateId & choice name (if present) on command submissions in the json API (#10005)
af144e8592 LF: Structure package loading errors (#9980)
3583fd1885 [JSON-API] Log source and path of incoming http requests & the response with info level (#10007)
e49c7c8db7 [In-memory fan-out] BufferedTransactionsReader implementation [DPP-413] [DPP-414] (#9882)
acb77336c8 Release 1.14 RC 2 (#10006)
3d2d4b9696 Don't duplicate log failed futures in the http json api (commandservice specific ones) (#9990)
0c32cf2968 Clarify the semantics of DIV_INT64 in LF spec. (#10003)
b79e739196 Fix Type (#10004)
998f106ee6 Drop pkgIds argument from Engine.validatePackages (#9986)
ae5010f3ef [Mutable state cache] Start cache from latest ledger head (#9998)
dd42a5a641 update maven_install_2.12.json (#9999)
66d38cd24d Storage query abstraction first sketch (#9959)
c75013622c Just print out the operator name for the operation logging in CommandService.scala (#9989)
b6faace335 Changing oracle backend from using varray to json array (#9943)
603a4a4b42 Add date to log statements in the HTTP JSON service (#9988)
836a82a6b6 Add proper error handling for missing contract keys (#9972)
85667af7be Address npm vulnerabilities (#9978)
d7d6e37785 LF: Structure Engine Errors. (#9971)
0adbbdcd58 update Daml Hub auth in create-daml-app (#9973)
997a7d449c Use symlinks on Windows (#9969)
e4258e61b1 docs: tweak Exception intro (#9967)
b2c4ef813d rotate release duty after 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9933)
de7a462082 Log call to submitAndWaitRequest with the command id provided in the log ctx (#9940)
630d021331 Collect build-event-stream (#9831)
26e199a719 Rerelease 1.11.2 snapshot (#9965)
8399039a8a Remove broken dep from release job (#9963)
e1db529162 DPP-390 configurable event decoding parallelism (#9931)
cc7af93afc [docs] Remove link to the #daml-contributors (#9928)
f7ac3ba784 Release 1.11.2 snapshot attempt 2 (#9962)
4634147ed1 Adds Indexer state to GRPC health checks [DPP-434] (#9951)
28c567b53d Release SDK 1.11.2 (#9957)
5f0f5756bf Define IncompleteTransaction for use by scenario-service (#9952)
b7fd8338c4 Increase `eventually` timeout when waiting for queue to be killed (#9955)
ca0fb16718 Fix collect_build_data job (#9956)
42030b25ed Skip Scala_2_12 job on releases (#9954)
5e1ddfbd5e LF: Fix specification of ArithmeticError (#9949)
3093a98b87 Forward port additional contract keys validation tests (#9944)
e541558f18 update NOTICES file (#9946)
db792a9774 Proxy ledger API health endpoint in JSON API health endpoint (#9945)
13767b5c5d Load the correct logback file for the http json service respecting the deployment situation (#9938)
4e49cf6814 `ledger-api-bench-tool` - Active contracts stream, completions stream [DPP-398, DPP-399] (#9857)
fce8e54469 release 1.14.0-snapshot.20210608.7123.0.3cb8d5c2 (#9932)
ab55fb2546 Handle module-prefixes in Java codegen (#9929)
7375e28271 Force resolution for css what to 5.0.1 (#9925)
```
Changelog:
```
- [JSON API] The template id & choice name (if present) are now logged on command submissions in the Json API (at trace level)
- [JSON-API] The source and the path for incoming http requests are now logged
- [JSON-API] The http response for a request is now logged
- [JSON API] Errors which arise from the CommandService are not logged twice anymore (thus reducing noise)
- [JSON API] log statements now include the date next to the time
- [HTTP-JSON] Calls which trigger a submitAndWaitRequest are logged with the command id provided in the log ctx
[Integration Kit] The state of the participant indexer can now be checked via the GRPC health endpoint
- [JSON API] The healthcheck endpoint on the JSON API now proxies the
health check endpoint on the underlying ledger. Previously it only
queried for ledger end to determine ledger health.
[HTTP-JSON]
- fixed that json log output could not be enabled via cli options except via usage of env vars
- [Integration Kit] - ledger-api-bench-tool - reading active contract streams
- [Integration Kit] - ledger-api-bench-tool - Reading completions stream
- [Java Codegen] The Java codegen will now pick up the
`module-prefixes` field from `daml.yaml` which can be used to handle
module name collisions between different DALFs.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
Commit log:
```
3cb8d5c2a7 Deprecate use of divulged contracts (#9930)
e9a2e2e7f2 Allow two different time formats as input for the metrics reporting interval (#9926)
1dc8472017 Fix variable name (#9927)
4037b1cd9e Add metrics to the http json service (#9923)
bd26c43767 LF: fix TransactionCoderSpec (#9922)
69ef6243cb Fix flaky InstrumentedSourceSpec test (#9921)
cbcec32e70 LF: clean up the remaining Exception TODOs (#9913)
537420942d Add dependency proto3-suite for Setup.hs (#9917)
c77324d041 skip fix_bazel_cache on forks (#9920)
ddf93fc413 Address TODOs in JdbcIndexer (#9911)
a0d3e80fd6 Add missing telemetryContext instances in the ledger-api-client's Ctx (#9916)
1e009b4ac2 Upgrade canton to 0.25.0-snapshot supporting lf-1.13 and enable/fix tests (#9915)
db1453640f LF: Freeze archive proto for LF 1.14 (#9912)
15f9dfb963 update compat versions for 1.14.0-snapshot.20210602.7086.0.f36f556b (#9900)
b4247c1511 [In-memory fan-out] BuffersUpdater implementation (#9858)
7e733c3cee DPP-390 remove appendonly todos (#9910)
512f1fd87d LF: Preview version of LF 1.14 (#9906)
2da94c3149 Add warning about using divulged contracts in pruning doc (#9883)
3dee3d042d LF: Fix encoder for Exception (#9904)
b22c04654d ledger-api-test-tool: Only wait for parties on participants under test. (#9903)
b8c02d65b7 Bump perf test for for #9871 (#9902)
4660e57e0c Fix typoed participant name in ExceptionsIT (#9901)
316069d047 Postgres batching refinements (#9898)
7bc925e4d2 LF: Factorize the logic for AST lookup (#9871)
2fa250f06a DPP-387 Remove migration todos (#9896)
65f9395cdd Try to fix windows skipping again (#9893)
850c7310e8 Include PackageMetadata in LF interface (#9892)
41266c35e2 Fix the formatting on the LF spec. (#9889)
dc79830239 DPP-393 Enable sandbox tests for the append-only schema (#9876)
3c83b77dd8 Parallel Ingestion renames (#9894)
6e48034edc Dpp 392 todo cleanup postgre batch (#9875)
08ed7b61ff rotate release duty after 1.14.0-snapshot.20210601.7081.0.7d716e6d (#9878)
081c54b411 Revert "Try to unflake JSON API failure tests (#9855)" (#9887)
df0f3ff7b0 maybe fix deleted release tags (#9891)
4b020dddc3 Release ledger-api-bench-tool (#9890)
d80191435f Release new snapshot (#9885)
4a4dde0f19 Implemented JdbcLedgerDaoTransactionLogUpdatesSpec (#9813)
```
Changelog:
```
- [Ledger API] Use of divulged contracts in later transactions is
deprecated. Divulged contracts were already incompatible with
pruning. We are working on features to handle the cases currently
covered by divulgence.
- for applications which support the --metrics-reporter-interval cli option, these now support both the java and scala duration string format (e.g. PT1M30S and 1.5min)
Small fixes to the doc
[HTTP/JSON API]
- metrics reporting can now be chosen via the command line option --metrics-reporter (currently hidden), valid options are console, csv://, graphite:// and prometheus://
- metrics reporting interval can also now be chosen via a command line option --metrics-reporting-interval (currently hidden)
[ledger api] Add warning about incompatibility between pruning and
divulged contracts to pruning docs.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@cocreature is in charge of this release.
Commit log:
```
7d716e6dba Try to unflake JSON API failure tests (#9855)
2d49939360 Fix the notion of transaction equivalence in the ledger model for exceptions. (#9850)
fd8f8a6ad9 Daml-lf suffixCid ignores V0 contract ids rather than erroring (#9867)
e38bb4a55d Make data-deps exception tests version-aware. (#9870)
94129b2254 Add tests for decoding of UpdateTryCatch (#9865)
1cc0906267 Adda test for an unhandled builtin exception to ExceptionTest.scala (#9866)
d7c60f5258 resources: Only mutate test mock objects inside `synchronized` blocks. (#9869)
3a0b2f9ccc remove 2x 8020 todo tags (#9868)
1595f06215 Special case _tryCatch @Update in LFConversion. (#9864)
c0359ac9ae Add trace context propagation in CommandClient.trackCommandsUnbounded [KVL-961] (#9833)
2a1e1ba1d5 Add a Foldable instance for Set (#9860)
590b8b3367 Reshuffle tests for AnyExceptionMessage (#9863)
89bd478949 Reenable AnyException tests in ExceptionTest.scala (#9861)
1573984673 Include rollback nodes in danglingRefGenNode (#9851)
e5d4553208 Remove flyway warning when using the append-only schema (#9840)
a4e33c9e1b Add metrics to parallel indexer (#9841)
972af2a1ae Test To/FromAnyException primitives in DecodeV1Spec (#9852)
5f4f019f49 update NOTICES file (#9859)
f5f2a8a3f7 Remove obsolete transaction todo (#9853)
8fc45432d1 Uniform configuration of the command service across all ledgers (#9839)
1f4a958432 ledger-api-bench-tool - single-value metrics (#9822)
6a1c7974dd ledger-api-bench-tool: TLS support (#9824)
55283472df Test TransactionsReader.getContractStateEvents within the JDBC DAO suite (#9849)
6af6c930a3 Fix behavior of Math.sqrt at 0 (#9818) (#9828)
7855ea50c2 Switch to Ubuntu 20.04 nodes for ghc lib (#9838)
c0fccfc671 Switch Scala version Bazel config (#9848)
e033277804 Switch to our own nodes for releases (#9845)
b6a89a6948 Bump ws to address security advisory (#9844)
25b7e544ce Only resolve template id once (#9837)
a645971e72 Enable mutable-contract-state-cache for ledger-on-SQL append-only tests (#9836)
dcd33a7b4b Add a section on exceptions to the Daml intro (#9832)
859cae3b46 [Mutable contract state cache] Tests for contract/key state new methods in ContractsReader (#9825)
ffe55863ff Track oldChild in PartialTransaction context (#9815)
6e17e2de75 [In memory fan-out] Transaction log updates stream implementation (#9792)
a0373c4855 Ledger model section on transaction normalization. (#9816)
f355931a59 Fix typeclass detection in data-dependencies (#9830)
9626a5952d Speedy: Clean SBuiltin (#9829)
269237ea76 Fix authorization in try contexts (#9823)
cb06fb3448 `ledger-api-bench-tool` - min consumption speed SLO [DPP-401] (#9808)
0868324c24 update NOTICES file (#9827)
a166e79aca Added maxlength information to id string (#9826)
63bc0d1a96 EventsBuffer implementation for in-memory fan-out for Ledger API serving (#9775)
0c12259c04 Speedy: Use SAny instead of SAnyException to reperesent AnyExcpetion (#9819)
6caa61c9d4 participant-state: rejection reason abstractions (#9764)
63c471fa79 Upgrade protobuf-java and scalapb. [KVL-938] (#9713)
f177c1a6e4 `ledger-api-bench-tool` - Moved classes to individual files [DPP-401] (#9803)
a3b10dc6f9 LF: fix parser (#9809)
9f55da813f update NOTICES file (#9812)
6846dbfaf8 update compat versions for 1.14.0-snapshot.20210526.7024.0.aedb9a82 (#9806)
1f021b25c6 LF: Drop Builtin Exceptions completly (#9790)
ea5b710d7d rotate release duty after 1.14.0-snapshot.20210525.7017.0.2710fad0 (#9795)
0c7b149068 Add reference docs for exceptions (#9807)
6b67ba06f2 LF: SBAnyExceptionMessage queries for unknown packages. (#9804)
fdab1e68d7 Parallelize daml-assistant integration tests. (#9802)
2e9c56e52c DPP-393 Activate JdbcLedgerDao suite for the append-only schema (#9793)
274fcaee10 release 1.14.0-snapshot.20210525.7017.0.2710fad0 (#9801)
3241ad6a41 `ledger-api-bench-tool` - updated metrics reporting model [DPP-400] (#9749)
```
Changelog:
```
Modify command line options of the ledgers based on the kvutils/app library:
- Remove max-commands-in-flight sub-option from the participant group.
- Add command service configuration options in line with sandbox and daml-on-sql: max-commands-in-flight, max-parallel-submissions, input-buffer-size
- [Integration Kit] - TLS support for the ledger-api-bench-tool
[Daml Compiler] Ensure that sqrt(0.0) == 0 and 0 ** 0 == 1
- [Integration Kit] - ledger-api-bench-tool - new consumption speed service-level objective for the ledger-api-bench-tool, configurable with a CLI parameter --min-consumption-speed
- Participant-state: Introduced versioned form of rejection reason
- [Integration Kit] - ledger-api-bench-tool - new parameter max-delay for specifying the service-level objective for max delay
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@SamirTalwar-DA is in charge of this release.
Commit log:
```
a6c7b86afc LF: Drop support of TO_QUOTED_TEXT_PARTY (#9721)
9d23b4337f Split DA.Internal.Exception.Types into three packages. (#9709)
66d039eb5b Upgrade gatling to version 3.5.1 (#9714)
ac9b8adc5c TODO Cleanup: store.backend package (#9693)
679ce3b7d3 LF: Release LF 1.13 (#9705)
73c479310d update NOTICES file (#9711)
f249eec24f drop per-commit compatibility_macos (#9730)
55c36cc93e Compiler: drop BEToTextBigNumeric in favor of BEToText (#9728)
2a2c745978 Implement name collision check for type synonyms. (#9723)
c7cc1bfdfa KVL-914 Add more unit tests for tracing (#9708)
e63e0cfb77 Upgrade canton to 0.24.0 (#9722)
0a1d00d1b3 Canonicalize junit output path (#9710)
d4fca0305b LF: Change error exception for Arithmetic builtins (#9692)
1e5664054d Release: Switch Andreas and me around in the release rotation. (#9719)
e50885caa8 Dpp 388 todo cleanup parallel indexer (#9690)
fc745f2224 LF: clean shifting BigNumeric builtin (#9704)
7a97d88f42 DA.BigNumeric: Split shift, add roundToNumeric. (#9702)
b1738c7202 Switch Scala default to 2.13 (#9699)
1edb110f99 Fix simplifier safety for AnyExceptionMessage (#9707)
0ab37b3774 Use new exception builtins in stdlib. (#9703)
7d472e178d Optimistically increase Patience for fixing flakiness in MutableCacheBackedContractStoreSpec (#9697)
69f51e4b5e speedy: Compile new AnyException primitives (#9700)
3666cf2f54 KVL-914 Add unit test and fix trace context propagation from config management service (#9694)
4e63299919 Simplify kvutils contract keys validation (#9628)
010e2b1b27 Fix client_server_build/test with port file (#9701)
6d9490c0bb Never create a rollback node containing pre-exception descendants. (#9679)
b19c8f2a7c Drop sanitize, add MagicHash in data-dependencies. (#9698)
c669a00883 Release SDK 1.13.1 (#9696)
395ff58eec Damlification of Bazel files (#9670)
9b53251c74 Differentiate between negative inputs & errors in contractKeyInputs (#9683)
cd9933328a runtime: Damlification of Scala files (#9668)
fb0551bdfd Fix blackduck flag names (#9695)
4b3b9ef165 Add race condition tests for exceptions (#9688)
927242be40 KVL-914 Add and rework unit tests for tracing (#9686)
6568336588 Lift constraint tuples up to a type synonym in data-dependencies. (#9687)
80f65b4734 LF: clean compilation of SToTextXXX builtins (#9682)
ed14f97528 Move race condition utilities to test tool infrastructure (#9685)
e5646a9c73 update compat versions for 1.13.1-snapshot.20210512.6834.1.5f532380 (#9681)
6629aab009 DEL-8479 tag LF libs for SDK 1.13 release (#9680)
55abd0b7f0 Add a data-dependencies test for user-defined exceptions (#9659)
142767c421 Split daml test-script code into a separate library (#9674)
d436686173 Rerelease 1.13.1 snapshot (#9678)
5d181c6ef4 Throw on internal errors instead of setting ptx to aborted (#9654)
ac2e285ed3 Release 1.13.1 snapshot (#9675)
af7f88fca5 restore natural join and avoid table aliasing in participant contracts queries (#9671)
625592e7b5 Rename CorrelationID to InstanceUUID as it has been used wrongly. (#9672)
638547fdde rotate release duty after 1.13.0-snapshot.20210511.6892.0.ca9e89b3 (#9648)
c282a09ffb KVL-914 Expose metrics-test-lib for the oem integration kit (#9662)
cc01e933e5 Update error message in interface reader. (#9657)
2dc8b8f1de Remove version field from rollback node. (#9627)
b58d30b9e3 skip Windows tests on release (#9656)
40f69ee5af fix release PR notifications (#9658)
691915353c Fix update metrics population in parallel indexer (#9655)
931274a14e Drop version check from TransactionPreprocessor (#9651)
4620851ba2 release 1.14.0-snapshot.20210511.6892.0.ca9e89b3 (#9647)
0d09db5c48 Update date on code owners in RELEASE.md (#9652)
e61e6b22e0 Release SDK 1.13.0 (#9650)
9e6d6f7cf0 Support append-only schema in Sandbox migration tests (#9637)
fb9a196431 update NOTICES file (#9649)
58d4281743 Document web socket event handlers of TS Stream type. (#9634)
111e1d3798 Fix sandbox classic package upload (#9633)
```
Changelog:
```
- LF: make LF 1.13 stable
Add support for BigNumeric
- Ledger API: Bump Ledger API version for LF 1.12
- [Daml Standard Library] `DA.BigNumeric.shift` has been split into
`DA.BigNumeric.shiftLeft` and `DA.BigNumeric.shiftRight`.
`DA.BigNumeric.roundToNumeric` is introduced, for rounding and
converting a BigNumeric to a Numeric in a single move.
- [Daml Compiler] Fixed a data-dependencies bug where functions in a data-dependency that used a constraint tuple constraint (e.g. `Template t`) could not be directly invoked.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
The first one ended up with a corrupted Windows installer. I couldn’t
find anything in the logs indicating why it would be broken and this
is the first time I’ve seen this so hopefully just rereleasing fixes it.
changelog_begin
changelog_end
* release 1.13.0-snapshot.20210511.6892.0.ca9e89b3
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@victormueller-da is in charge of this release.
Commit log:
```
ca9e89b3da check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
4a460973a2 Block concurrent SDK installations. (#9640)
c757b61516 Check byKey in transaction validation (#9641)
4f0ceff7a2Fix#7170 (#9618)
d3106682dd Add a REPL for each Scala target, for debugging. (#9643)
22b36b0b01 Include byKey in LF transaction spec (#9642)
ca027e3dde Add Transaction.contractKeyInputs to inputs of a transaction (#9631)
c3d79d4656 Allow OverloadedLists in data-dependencies (#9636)
f8b35fef9c Cleans stateUpdates of ReadWriteServiceBridge (#9630)
3750b16a11 Serializalize byKey flag in Daml-LF transactions (#9632)
3b59f8ce87 ledger-api-bench-tool - prototype [DPP-370] (#9606)
f7b33d93fb Authorization for daml script exports (#9629)
0d1f3db8a4 LF: refactor builtin exceptions in Speedy (#9605)
d6d01b04ef Swap order SEScopeExercise and SBUBeginExercise (#9621)
6c919b3877 sandbox-on-x: Support `beginAfter` in state updates. (#9624)
26a8011ba5 TLS for Daml Script exports (#9626)
9242540022 Make error throw a GeneralError. (#9613)
bdcf0ecb66 update NOTICES file (#9623)
4c1fbeb194 Add duplicate contract key checks to Speedy (#9607)
871279f3a6 LF: extends LF command with Fetch and Lookup (#9587)
0acc4f115c Patch old Bazel derivation (#9622)
b082274885 Address vulnerabilities in Navigator (#9617)
560653a5d7 Unwind partial transaction on mustFail (#9608)
f51145cb88 Support mapping from old to new party ids in daml exports (#9591)
89e46bf90b Address security vulnerabilities in //compiler/daml-extension (#9615)
fd62671e0b Introduce SINCE-LF-FEATURE in integration tests. (#9616)
48939b513b Address vulnerabilities in //language-support/ts/packages (#9614)
cf59246d44 Add support for daml exceptions for append-only indexer (#9609)
ab29f7c07c Move activeness check of globalKeyInputs into archive (#9610)
5c28de36cb Upgrade grunt to address cve (#9611)
22ba5fddd2 Remove builtin exception types from protobuf and ASTs. (#9595)
b09a95fb6f Adapt indexer to empty divulgences (#9598)
2fc7489e44 Filter divulgence to an empty set of parties (#9600)
ad45213b66 participant-integration-api: Avoid the serial EC in integration testing. (#9603)
f742a4334e Dpp 336 sandbox classic on append only (#9561)
e68bc0dff0 Mark reset service tests as flaky (#9602)
2176173ba2 Remove 1.dev-only things from LF 1.13 protofile. (#9599)
03034ec3bf DPP-359 Add indexer flow level benchmark (#9509)
5aa54017eb Drop todo for conversion of exception primitives (#9597)
968b5d8d39 KVL-921 Expose opentelemetry Context from Telemetry and TelemetryContext (#9573)
112c387e5a Refactor out setGlobalLogLevel into ContextualLogging (#9592)
e584fecd3c Drop com.daml.lf.engine.Event.collectEvents (#9596)
80b07da309 Only archive a key if it was brought into scope before (#9546)
7ea9340d64 Support old bash in daml-sdk-head (#9593)
45fbdefb06 Handle rollback nodes in protoNodeInfo (#9589)
42d189dbbe Support exceptions in Daml-LF encoder (#9590)
9c3913a3d6 LF: SBUKeyBuiltin clean up. (#9572)
5128206ebf Ledger API Server: Named threadpools (#9588)
26a53d8469 Add logging command line option to ledger http service (#9581)
409833f850 Address rollback todo in ancient migration (#9583)
a27b2c56bc Address more exception todos (#9582)
2040556dbb Support rollback nodes in replay adapter (#9584)
192a77a6d8 update compat versions for 1.13.0-snapshot.20210504.6833.0.9ae787d0 (#9575)
81d508b775 LF: make SBuiltins pure (#9580)
de80a6dc60 Drop ValueBuiltinException (#9576)
1c456be79d Release yet another 1.13 snapshot (#9574)
800989145b Disable migration tests on macos on PRs (#9571)
2e93de79a4 Fix recording exceptions in Spans, add unit tests [KVL-874] (#9544)
121ded3565 Accept a comma-separated list of parties for daml ledger export (#9568)
```
Changelog:
```
- [Daml Assistant] The assistant will now avoid
installing SDK versions concurrently, waiting
for the previous installation to finish before
starting the next installation (if still
necessary). This fixes a bug where the SDK
would become corrupted because two
installations were started concurrently.
http-json:
- add contextual id as logging context to distinguish different application runs in logs
- add request id as logging context to distinguish different http requests within an application run
- add for non-static endpoints trace logs which show how long processing it took in ns
- [Integration Kit] - Created the ledger-api-bench-tool prototype for benchmarking ledger transaction streaming capabilities
* [Daml export] Users can now define a mapping from parties in the
original ledger state to parties to be used when recontructing the
ledger state. The ``parties`` component of the argument to the
generated export script now takes a mapping from old party name to new
party name.
- [Ledger HTTP Json Service] Logging now also tells service name if log level was changed.
- [Ledger HTTP Json service] Logging can now be configured via the `--log-level` cli argument. Valid values are `error`, `warn`, `info` (default), `debug`, `trace`
```
CHANGELOG_BEGIN
CHANGELOG_END
* Release snapshot 1.14.0
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: victor.mueller@digitalasset.com <mueller.vpr@gmail.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@robin-da is in charge of this release.
Commit log:
```
90c5ce703a Optimize mapOptional and add more efficient findOptional (#9214)
1e0c67c306 LF: Add RoundingMode and BigNumeric in Haskell Ast (#9215)
e492603808 More manual list fusion in stdlib (#9217)
90aa84193f tuesday notif: newlines (#9213)
51c5eac66e LF: Add BigNumeric to archive proto (#9210)
615f4e4bae Support passing sandbox port via --sandbox-option --port (#9208)
349e0a0bef LF: fix package name in LF 1.12 proto snapshot (#9211)
b7aedb477b Remove contract arguments from localContracts (#9207)
c1cf2756a8 add rollback node to transactions (#9178)
2551b2d615 LF: Add BigNumeric and Rounding to Scala AST (#9205)
be4dbd971d update compat versions for 1.11.1 (#9172)
0f9ab428ee kvutils: Remove ledger dump functionality (#9204)
f89aa294e5 damlc pkg: use cache for already present packages (#9193)
7a41ee6ce0 Cache contract fetches in speedy (#9192)
644561fe23 Bump jinja2 from 2.11.2 to 2.11.3 in /docs/scripts (#9198)
bf76e3c84a update NOTICES file (#9201)
4d90d02740 LF: drop useless field arity in Ast Builtin (#9194)
a93f89b6f8 Factor out an Oracle test fixture (#9191)
51f495e758 KVL-203 Share participant state integration test harness (#9143)
440eead6c4 Make LedgerDao a pure interface (#9186)
617e6f58e8 update NOTICES file (#9190)
587a9c95d0 Recognize (some) exerciseByKey commands in Daml script dumps (#9185)
ad94c47605 ci/docker: sync with hub (#9184)
081623b493 fix: bad package id validation (#9183)
f0c8b1240d Integrate non-repudiation into HTTP JSON API (#9180)
dbd8806848 Distinguish all four types of submits (#9179)
8c64f120da pkgid data deps (#9153)
d5ca22815d Recognize simple createAndExercise commands (#9154)
6c9679f511 Add documentation for the Daml Profiler (#9174)
a6da995ecb use representational variance for generated Scala classes (#8879)
47e0a223e7 rotate release duty after 1.12.0-snapshot.20210316.6523.0.b382fc45 (#9161)
1ea00976f0 skip compat tests on release (#9169)
c26c349c8b Generate exception instances from syntax. (#9140)
9d175ce547 Document profiling internals (#9168)
5e97c573e0 Add a test for create and exercise events in the profiler (#9151)
75f7688cae Expose unhandled exception values and types (#9158)
e4d5799907 weekly release (#9167)
370f290d3d Handle createAndExecuteCmd (#9150)
af0cb30828 Release SDK 1.11.1 (#9162)
```
Changelog:
```
- [Daml Assistant] The sandbox port can now also be configured via
`--sandbox-option=--port=12345` instead of `--sandbox-port`. Other
tools like Navigator, the JSON API and Daml Script will pick up the
modified port automatically.
- [JSON API] The JAR for the HTTP JSON API is no longer published on
Maven, use the fat JAR as indicated in the documentation.
- [Daml Profiler] Daml Connect EE now includes a profiler which can be
used to profile Daml execution time.
- [Scala Codegen] Type parameters used in GenMap keys are now supported.
See `issue #8879 <https://github.com/digital-asset/daml/pull/8879>`__.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
Commit log:
```
b382fc45ac update areas to alert @S11001001 of changes (#9156)
bbd239b0d3 Unstringify profiler labels (#9147)
3870f84534 Drop profiler from Sandbox CE (#9152)
691edeacf2 ci: fix cache cleanup (#9137)
98c2651998 Support fetching SDK EE tarball in the assistant (#9146)
9b2158508b Add new variant to Value.scala for builtin-exceptions. (#9084)
75cec502ac Add tests for unmangleLenient (#9148)
1df2270cc9 typo (#9145)
6018697fb4 Make async commit configurable and add "LOCAL" async commit option (#9144)
ab90c437a7 update compat versions for 1.11.1-snapshot.20210312.6429.1.7cd6380e (#9135)
498fcc66b4 KV specific LF library (#9100)
c327476da9 Upgrade msys packages (#9139)
089a11443c release job: remove extra bash-lib (#9136)
b0948b417b release 1.11.1-snapshot, take n+1 (#9134)
5dddf0aead Add missing checkout step (#9133)
7669d8c88c daml pkg: split installation of deps and package db inititialization (#9056)
d5e0d6ca00 Release 1.11.1 snapshot (#9130)
7430590c1d Only track cids referenced in root events (#9131)
d4bdb12862 Add pruning stabilization to ledger API changelog (#9126)
ab7e7d9d94 Ignore fetch and lookup-by-key nodes for state update comparison [KVL-854] (#9070)
208f6c1aab Fix javadocs & sources in daml-sdk-head (#9125)
9320a213db [Integration Kit, Ledger API] Remove hint about early access for participant pruning. (#9120)
1c5e64cb82 Fix release notes link in docs (#9122)
ed746976a3 Apply new logo to create-daml-app template (#9105)
7370313bfa Release yet another snapshot (#9118)
```
Changelog:
```
- [Daml Assistant] The assistant now supports an `artifactory-api-key`
field in `daml-config.yaml`. If you have a license for Daml Connect
EE you can specify this and the assistant will automatically fetch
the EE edition which provides additional functionality.
[Integration Kit, Ledger API] Remove hint about early access for participant pruning. It's stable now.
[SDK Assistant] New Daml logo on create-daml-app template
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Bumping the snapshot release number as the fix was on the release
process on main (i.e. not change to the target branch/commit).
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.11.0-snapshot.20210309.6463.0.f7abca91
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@S11001001 is in charge of this release.
Commit log:
```
f7abca919f fix readme in create-daml-app: add npm install (#9060)
e04bd91eda Move Daml Profiler to EE version of sandbox/sandbox-classic (#9054)
7859bc13e1 inline help: scripts JSON API clarification (#9057)
642385a692 Upgrade elliptic dependency to address vulnerabilities (#9055)
f2b9f118cf Check that only the command signed payload goes through (#9053)
5197374cab Build SDK EE tarball (#9049)
1bec2116ba Add tests for client bindings (#9036)
27fd9327f6 Disclose transactions in the flat transactions stream only to the stakeholders - test case [DPP-276] (#9028)
45b33757b2 Introduce a helper to limit signing to commands (#9050)
0c4d8ac19c Speedy: rollback within exercise (#9040)
caa023b72e ci/cron/check: remove dade-assist calls (#9048)
85f0965ab9 LF: fix archive decoder for GREATER_DATE (#9047)
121534c54d ci/cron/check: low-hanging perf improvement (#9042)
fb0bd82740 update NOTICES file (#9045)
4fd42a6772 reduce noise on daily tests (#9039)
5d88c08832 Bump ghcide (#9041)
41d62ea5c0 send CI failures on separate channel (#9038)
e26d006f6f slack: reduce @here mentions (#9037)
ae110361f0 Add more shared infrastructure for non-repudiation testing (#9035)
75140744aa Speedy: building transactions with rollback (#8983)
98410e7c7f update compat versions for 1.11.0-snapshot.20210304.6422.0.d3d5042a (#9032)
0f1d2a9c47 Upgrade akka to 2.6.13 (#9026)
b1a59d0c70 Release SDK 1.11.0 RC 2 (#9031)
41e049339b Fix --extract option in ledger api test tool (#9029)
0281b442b8 -Werror all the things (#9027)
2c08586d33 Expose Daml stacktraces for Daml Script errors (#9025)
c7ee410fa3 Use DA.Map in triggers if available (#9023)
d347934db3 Make non-repudiation PostgreSQL back-end certificate adding idempotent (#9024)
eaf7fb6f63 Add non-repudiation API (#8982)
5a3c95251c Document latency of Windows ad-hoc machine software installation (#9022)
3758b57a87 pkg deps: Move all dalfs to package dependency directory (#9002)
f70d3391ef update compat versions for 1.10.2 (#9016)
e5421288d9 Improve errors on duplicate record field names (#9010)
65fbcfe8e9 Move execution of commands out of Runner (#9009)
2688ad6f0d autorelease: improved PR message (#9008)
185e91646d Release 1.10.2 (#9006)
4ed3f1a6de Cut a snapshot release for the 1.8.0 special patch (#8998)
ba1456fd5a rotate release duty after 1.11.0-snapshot.20210302.6414.0.72870630 (#8992)
259bf682c4 compat: fix killing stale sandbox(es) (#9003)
32dc8b007b Include stack traces for all ScriptF commands (#8999)
1525957c7a release 1.11.0-snapshot.20210302.6414.0.72870630 (#8991)
926949e503 Use single party submit/submitTree where appropriate (#8995)
```
Changelog:
```
- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
produced errors for warnings produced by `-Wmissing-signatures` even
if the user did not explicitly enable this.
- [Daml Script] When running Daml Script on the command line you will
now see a Daml stacktrace on failures to interact with the ledger
which makes it significantly easier to track down which of the calls
fails. By default, you will only get the callsite of functions like
`submit`. To extend the stack trace, add `HasCallStack` constraints
to functions and those will also be included.
- [Triggers] The trigger library now uses `DA.Map` instead of the
deprecated `DA.Next.Map` if the targeted Daml-LF version supports it.
This is a breaking change: Code that interfaced with the triggers
library using `DA.Next.Map`, e.g. with
`Daml.Trigger.getCommandsInFlight` or `Daml.Trigger.Assert.testRule`,
will need to be changed to use `DA.Map` instead.
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.12 not 1.11
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* cut a snapshot release for the 1.8.0 special patch
CHANGELOG_BEGIN
CHANGELOG_END
* update for #9000
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@hurryabit is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* release 1.9.0-snapshot.20210119.6103.0.cdcf090b
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump version, changing prefix from 1.9 to 1.10
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Release yet another 1.9.0 snapshot
This includes #8490 which fixes triggers with LF 1.11. Since LF 1.11
is mentioned very prominently in our release notes, we don’t want to
break triggers there.
changelog_begin
changelog_end
* Release: Update to commit 9ed787cb3d.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* release 1.9.0-snapshot.20210105.5984.0.c68ba110
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@robin-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump commit to include #8402
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This is not intended for general consumption.
This reverts the following PRs, in reverse order:
- #8082
- #8146
- #8153
- #8180
- #8194
CHANGELOG_BEGIN
- [Integration Kit] Reverted a fix to the kvutils input state that
caused performance degradation on some ledgers. This is a temporary
measure that we do not expect to be used outside of very specific
circumstances, as it can trade correctness for performance if the
semantics are not completely understood.
CHANGELOG_END
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@cocreature is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
There is no other way to test this code than to make a release. This is
expected to be identical to 1.7.0-snapshot.20201103.5565.0.e75d42dd,
with the version number being the only difference.
CHANGELOG_BEGIN
CHANGELOG_END
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@aherrmann-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Release another 1.6.0 snapshot
This includes the backported timeout bump from 5cb20c1145.
changelog_begin
changelog_end
* add gsg fix
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@hurryabit is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.6.0-snapshot.20200929.5303.0.f1e58206
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump release commit
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@S11001001 is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Note that following the changed process for release notes which now
just point to the blog, this does not include an update to the release
notes in the rst file.
changelog_begin
changelog_end
* release 1.5.0-snapshot.20200901.5116.0.4460cb5e
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@rohanjr is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.5.0-snapshot.20200902.5118.0.2b3cf1b3
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@leo-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Release 1.3.0 snapshot
This includes the fix for the indexer, a fix for the stackoverflow and
unpublishes daml-on-sql from GH releases.
changelog_begin
changelog_end
* Bump commit
changelog_begin
changelog_end
Note for reviewers: This is a hotfix for a snapshot. I’m not going to
into why we need this here. As for the version number we decided that
the least confusing option here is the following:
Use the sha and number of commits as usual but use the date of the
original snapshot that we are patching here.
changelog_begin
changelog_end
trigger all releases from master
The 1.1.0 release went wrong and we had to trash it and release 1.1.1
instead. This is an attempt at identifying and correcting the root
cause behind that incident.
To understand the situation, we need to know how releases worked before
1.0. We had a one-line file called `LATEST` that specifies the git SHA and
version tag for the latest release. A change to that file triggered a
release with the specified release tag, built from the source tree of
the specified commit. The `LATEST` file looked something like:
```
f050da78c9 1.0.0-snapshot.20200411.3905.0.f050da78
```
To mark a release as stable, we would change it to look like this:
```
f050da78c9 1.0.0
```
i.e. simply drop the `-snapshot...` suffix. Even though the commit (and
thus the entire source tree we build from) is the same, we would need to
rebuild almost all of our release artifacts, as they embed the version
tag in various places and ways. That worked well as long as we could
assume we were doing trunk-based development, i.e. all releases would
always come from the same (`master`) branch.
When we released 1.0, and started work on 1.1, we had a few bug reports
for 1.0 that we decided should be resolved in a point release. We
decided that the best way to handle that would be to have a branch
starting on the release commit for 1.0, and then backport patches from
`master` to that branch. We adapted our build process to also watch the
`release/1.0.x` branch and, in particular, trigger a new release build if
the `LATEST` file in that branch changed. That worked well.
The plan going forward was to keep doing regular snapshot releases from
the `master` branch, and create support, point releases ("patch" releases
in semver) from dedicated branches.
On April 30, we made a snapshot release as an RC for 1.1.0, by changing
the `LATEST` file in the `master` branch. That release was built on commit
681c862d. On May 6, we decided to take a new snapshot as the RC for
1.1.0; we changed `LATEST` in `master` to designate 7e448d81 as the new
latest release.
On May 11, we noticed an issue that broke our builds. Without going into
details, an external artifact we depend on had changed in incompatible
ways. After fixing that on `master`, we reasoned that this would also
break the build of the final 1.1.0 release if we just tried to build
7e448d81 again. But as the target release date was May 13, we did not
want to take a new snapshot after that fix, as that would have included
one more week of work in the release, and given us no time to test it.
So we did what we did for the 1.0 branch, as it had worked well: we
created a branch that forked from `master` at commit 7e448d81 and called
it `release/1.1.x`, then cherry-picked the one fix to our build process to
work around the broken download. When the time came to make the final
1.1.0 build on May 13, we naturally picked the `LATEST` file from the
`release/1.1.x` branch and dropped the `-snapshot...` suffix. Importantly,
we did not need to update the target commit to include the "broken
download" fix as, in the meantime, the internet had fixed itself, and we
thus reasoned we should go for the exact code of the RC rather than
include an unnecessary, albeit seemingly harmless, change.
Everything went well with the release process. Tests went well too. Then
we got a report that an application that worked against the latest RC
broke with the final 1.1.0. The issue was that we had built the wrong
commit: by branching off at the point of the _target_ commit for the
latest snapshot, we did not have the change to the `LATEST` file that
designated that commit as the target. So the `LATEST` file in
`release/1.1.x` was still pointing to 681c862d.
I believe the root cause for this issue is the fact that we have
scattered our release process over multiple branches, meaning there is
no linear history of what was released and we are relying on people
being able to mentally manage multiple timelines. Therefore, I propose
to fix our release process so this should not happen again by
linearizing the release process, i.e. getting back to a situation where
all releases are made from a single branch, `master`.
Because we do want to be able to release _for_ multiple release branches
(to provide backports and bugfixes), we still need some way to
accommodate that. Having a single `LATEST` file in the same format as
before would not really work well: keeping track of interleaved release
streams on a single file would not really be easier than keeping track
of multiple branches.
My proposed solution is to instead have a multiline LATEST file, so that
all the release branch "tips" can be observed at the same time, and, as
long as we take care to only advance one release branch at a time, we
can easily keep track of each of them. This is what this PR does.
This required a few changes to our release process. Most notably:
- Obviously, as this is the main point of this PR, the build process has
once again been restricted to only trigger new releases from the
`master` branch.
- As our CI machinery cannot easily be made to produce multiple releases
from a single build, the `check_for_release` step will only recognize
a commit as a release trigger if it changes a single line in the
`LATEST` file. This restriction comes in addition to the existing one
that a release commit is only allowed to change either just the
`LATEST` file or both the `LATEST` and
`docs/source/support/release-notes.rst` files.
- The docs publication process has been changed to update _all_
published versions to display the _latest_ release notes page. This
means that the release notes page will always show you all published
versions, regardless of which version of the documentation you're
looking at. This also means that interleaving release notes correctly on
that page is a manual exercise.
- As per the intention of the new process, the `LATEST` file has been
updated to contained all existing post-1.0 stable releases. It should
also include all existing snapshot releases should we have more than one
at a time (say, should we discover an issue with 1.1.1 that required us
to work on a 1.1.2).
- The `release.sh` script has been dramatically simplified as I felt it
was trying to do too much and porting its existing functionality to a
multi-line `LATEST` file would be too hard.
CHANGELOG_BEGIN
CHANGELOG_END
This is not intended to be the next stable release, I just want to
test the Bintray removal and publishing the protobufs to github releases.
changelog_begin
changelog_end
Let's see how far we get with #4745.
It's a bit of a shame I can't retry the same commit multiple times.
Maybe I should have accounted for that in the version format...
CHANGELOG_BEGIN
CHANGELOG_END
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
