This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
Commit log:
```
b4d00317b4 detect unsynchronized contract table and retry (#10617)
3d779cfe12 [Mutable state cache] Fix initialization offset (#11024)
5458aa890c Switch on NonUnitStatements warning in daml-lf/transaction (#11048)
9641fd5f83 auth middleware: no print secret (#11050)
a885f52c4d [DPP-417] [DPP-595] Add error code version switching mechanism (#11035)
12e0c72d5c fix blackduck logic (#11049)
eb87b3439b kvutils: Add the logging context for ledger state operations. (#11030)
b7daa5f7d8 Address remaining dependabot alerts (#11045)
5e43f8c703 es: drop jobs-* indices (#10857)
03203b7360 Define encoding/decoding for module imports (#11036)
57a15972c7 Setting timeoutToleranceMillis to 10 minutes to prevent flakiness (#11043)
df59f3fe8e Fix Navigator dependabot alerts (#11044)
6bf45a344a Upgrade Navigator to Webpack 5 (#11040)
80e217e11f [DPP-622] Add conformance tests that verifies TLSv1.0 and TLSv1 are disabled. (#10983)
626e1fbd7d Small lf value.cids regression fix (found by canton unit tests) (#11032)
a4629a4450 KV: Ignore daml_lf_1.proto when checking for KV protobuf compatibility (#11021)
ee9be65d68 kvutils: Add metadata to `Err` [KVL-1032] (#10992)
0d3ae6e14c interface methods: Haskell Typechecker (#11028)
e36eb46f59 Resolve `set-value` to 4.0.1 and above (#11029)
4075624cfc interface methods: Haskell AST (#11018)
7c1fd50469 Bump grunt-browserify (#11026)
5f3f5824f2 Upgrade webpack-dev-server in Navigator (#11025)
91be1e1659 Drop matchdep dependency from docs build (#11023)
fe9aeffeaf Increase es disk size (#11019)
eac7963b87 LF: Refactor ProtoTest.scala (#11020)
e79a30aa80 For the client binding propagate the full original completion [KVL-1112] (#10879)
59ad995282 fix buf check (#11014)
abc3e6691c Increase the tolarance for handover of control in HaCoordinatorSpec (#10997)
19b2bf477f LF: Cosmetic clean-up in the Speedy Compiler (#11015)
cb0e41f101 LF: Add interface support to the Preprocessor (#11013)
c33297c5fe Remove `transactionNormalization` flag. (#11010)
f5d2135f97 Check protobuf compatibility of `main` and PR commits w.r.t. previous stable release [KVL-1109] (#10950)
bf8b75dab5 interface methods: Add protobuf definitions. (#11005)
88e1430a42 Make LargeTransactionTest use ValueEnricher, so it can work with normalized transactions coming out of the engine. (#11003)
a043926c30 rotate release duty after 1.17.0-snapshot.20210921.7889.0.1b473c2b (#10972)
35666caae8 Add MINIMAL pragma for Additive type class (#11001)
8de162baec [DPP-586] Upgrade to netty 4.1.67.Final and netty-tcnative-boringssl-static 2.0.40.Final (#10956)
871d03ba65 release 1.18.0-snapshot.20210922.7908.0.ced4a272 (#10998)
721575ea73 [JSON-API] Postgres perf job (#10986)
f2d9f07417 Release RC2 for SDK 1.17.0 (#10996)
```
Changelog:
```
- [JSON API] Under rare conditions, a multi-template query backed by database
could have an ACS portion that doesn't match its transaction stream, if
updated concurrently. This conditions is now checked and accounted for.
See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
- The OAuth2 Middleware now obfuscates its Client Secret when logging
its config.
- [Integration Kit] We have added ``loggingContext`` as an implicit
parameter to more _kvutils_ trait methods. Implementors may need to do
the same in their trait implementations. This can make it easier to
log with the appropriate context.
java-client-bindings - the original full completion is included in the `CompletionResponse` when available
Daml on SQL, Integration Kit, Sandbox: Drop support for TLS 1.0 and 1.1 in Ledger API.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* enumerating out-of-sync offsets at the DB level
* cleanup in lastOffset
* write the latest-requested-or-read offset when catching up
- Writing only the latest-read, as before, would imply unsynced offsets
that are actually well-synced. This puts the DB in a more uniform
state, i.e. it should actually reflect the single value that the
fetchAndPersist loop tries to catch everything up to.
* detecting lagging offsets from the unsynced-offsets set
- Treating every unsynced offset as a lag would make us needlessly retry
perfectly synchronized query results.
* add Foldable1 derived from Foldable for NonEmpty
* nicer version of the unsynced function
* ConnectionIO scalaz monad
* rename Offset.ordering to `Offset ordering` so it can be imported verbatim
* finish aggregating in the lag-detector function, compiles
* port sjd
* XTag, a scalaz 7.3-derived tag to allow stacked tags
* make the complicated aggregation properly testable
* extra semantic corner cases I didn't think of
* tests for laggingOffsets
* a way to rerun queries if the laggingOffsets check reveals inconsistency
* if bookmark is ever different, we always have to rerun anyway
* boolean blindness
* incorporate laggingOffsets into fetchAndPersistBracket
* split fetchAndPersist from getTermination and clean up its arguments
* just compose functors
* add looping to fetchAndPersistBracket
* more mvo tests
* test unsyncedOffsets, too
* Lagginess collector
* supply more likely actual data with mvo tests; don't trust Java equals
* rework minimumViableOffsets to track sync states across template IDs
* extra note
* fix the tests to work against the stricter mvo
* move surrogatesToDomains call
* more tests for lagginess accumulator
* add changelog
CHANGELOG_BEGIN
- [JSON API] Under rare conditions, a multi-template query backed by database
could have an ACS portion that doesn't match its transaction stream, if
updated concurrently. This conditions is now checked and accounted for.
See `issue #10617 <https://github.com/digital-asset/daml/pull/10617>`__.
CHANGELOG_END
* port toSeq to Scala 2.12
* handle a corner case with offsets being too close to expected values
* didn't need XTag
* Add NonUnitStatements warning in daml-lf/transaction & fix using discard or similar.
CHANGELOG_BEGIN
CHANGELOG_END
* address comment
* prefer parens to braces for single statement map/foreach bodies
At the moment, the OAuth2 Middleware prints its config at startup.
Unfortunately that config includes a secret, which presumably we'd
prefer not to show.
CHANGELOG_BEGIN
- The OAuth2 Middleware now obfuscates its Client Secret when logging
its config.
CHANGELOG_END
Adding:
- a mechanism to switch between error code versions: `ErrorCodesVersionSwitcher`,
- the corresponding CLI flag `--use-error-codes-v2` for sandbox-classic, sandbox and ledger-on-X.
---
As of this PR:
- `ErrorCodesVersionSwitcher` is never used in production code (apart from a no-op usage in `ApiTransactionService` to prove the point),
- `--use-error-codes-v2` flag is set to hidden (until new error have been adopted).
These will be used in the subsequent stories from the same epic: https://digitalasset.atlassian.net/browse/DPP-417
We believe the Blakduck logic is currently faulty. We have had a
violation on an NPM dependency, and Blackduck keeps reporting it despite
our having removed the dependency.
We believe that what is happening is that, in the first step of
checking, we udpate the Haskell dependencies, _and then check the
validity of the whole project_, which includes the NPM deps. Because
that fails, we never get to the step where we actually update the NPM
deps, and Blackduck is stuck forever.
The solution is to not fail on violations for the Haskell update steps.
Haskell deps are still checked in the second step, because, again, it is
checking the whole project.
CHANGELOG_BEGIN
CHANGELOG_END
We do some logging in proprietary ledger drivers which would benefit
from correct propagation of the context.
CHANGELOG_BEGIN
- [Integration Kit] We have added ``loggingContext`` as an implicit
parameter to more _kvutils_ trait methods. Implementors may need to do
the same in their trait implementations. This can make it easier to
log with the appropriate context.
CHANGELOG_END
We are currently ingesting Bazel events in two forms:
In the `events-*` indices, each Bazel event is recorded as a separate ES
object, with the corresponding job name as a field that can serve to
aggregate all of the events for a given job.
In the `jobs-*` indices, each job is ingested as a single (composite) ES
object, with the individual events as elements in a list-type field.
When I set up the cluster, I wasn't sure which one would be more useful,
so I included both. We now have a bit more usage experience and it turns
out the `events-*` form is the only one we use, so I think we should
stop ingesting evrything twice and from now on create only the
`events-*` ones.
CHANGELOG_BEGIN
CHANGELOG_END
* Define encoding/decoding for module imports
First step towards closing #10773
changelog_begin
changelog_end
* Update compiler/damlc/daml-lf-conversion/src/DA/Daml/LFConversion/MetadataEncoding.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
I used a selective override for strip-ansi because html-webpack-plugin
cannot handle 7.0.0 and I don’t want to downgrade it in other places
where we are already using 7.0.0.
changelog_begin
changelog_end
* interface methods: Haskell AST for methods
Part of #11006. This leaves typechecker and LF conversion for later, on
the haskell side.
changelog_begin
changelog_end
* Forgot ECallInterface in DecodeV1
* fix a test
Tested manually that it worked. It didn’t work before actually since
the `webpack serve` switch was needed even for the old version but now
it does.
Unfortunately this still doesn’t get rid of the dependency on setvalue.
changelog_begin
changelog_end
We really don’t need 600 lines of mostly deprecated and unmaintained
dependencies which get flagged by blackduck & dependabot if we can
inline this into a static array which we never changed since we first
added the docs.
changelog_begin
changelog_end
Disks are currently at 75% fullness, so seems like a good idea to bump a
bit. #10857 should reduce our needs, too, so this should last us a
while.
CHANGELOG_BEGIN
CHANGELOG_END
* For the client binding propagate the full original completion.
Because more fields are added to the completion it gets more difficult to deconstruct it into our own models while keeping the same external API. Because of this, we are propagating the full completion as well
CHANGELOG_BEGIN
java-client-bindings - the original full completion is included in the `CompletionResponse` when available
CHANGELOG_END
* Check protobuf compatibility of release commits w.r.t. previous stable release
CHANGELOG_BEGIN
CHANGELOG_END
* Remove blank line
* Don't persist credentials
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* check-protobuf-against-stable.sh: SRC_DIR -> PROJECT ROOT + simplify
* Don't set LATEST_STABLE global in a function
* Simplify by using only the main work tree
* Simplify further as the check will be only run from `main`
* Move the check to `ci/build.yml` so that it is also run on PRs
* Enter the development environment to use tools
* Make variables read-only
* Support release branches and PRs targeting them
* Fix and document the reference tag finding logic
* Fix SYSTEM_PULLREQUEST_TARGETBRANCH and print it
* Don't log the source branch
* Fix comment formatting
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Enable Slack integration
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Don't check if the branch is a release one
...as the check won't be run on release branches.
* Add compatibility_stable_protobuf to collect_build_data
* Do not activate dev-env globally but only in sub-shells
* Add an explanation about why the check is not run on release branch commits
* Simplify further by leveraging `buf`'s ability to compare against branches
* Use `buf`'s `tag` locator instead of `branch`
* Split buf checks by module and remove previous manual check
* Explain how to run locally
* Use more future-proof WIRE_JSON for participant-integration-api
Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
* Use stricter FILE for the ledger gRPC API
* Propose an explanation for WIRE in kvutils
* Fix comment typo
* Re-introduce linting configuration for kvutils
* Simplify explanation for KVUtils' breaking check rule
* Remove extra (C) header from 3rd-party proto
* Don't touch the copyright of google/rpc/status.proto
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Simon Meier <meiersi-da@users.noreply.github.com>
* interface methods: Add protobuf
Adds protobuf definitions for interface methods and calling them.
Encoders/decoders just ignore the extra stuff or error out.
Part of #10810 (maybe)
changelog_begin
changelog_end
* Update issue numbers where appropriate
* update stable protos
(Also: generalize type of `enrichTransaction` to work for _submitted_ and _committed_ transactions & adapt existing callers)
This change paves the way to remove the `transactionNormalization` flag, and always normalize transactions coming out of the engine.
CHANGELOG_BEGIN
CHANGELOG_END
@garyverhaegen-da is taking care of 1.17.0-snapshot.20210921.7889.0.1b473c2b (#10971), so they get pushed back to the end of the line.
Please do not merge this before #10971.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Add MINIMAL pragma for Additive type class
This ensures that the compiler gives an error if both (-) and negate are missing in an instance
Fixes#11000
changelog_begin
changelog_end
* Add test case to ensure incomplete Additive instance declaration triggers error message
@garyverhaegen-da is in charge of this release.
This supersedes the failed #10971 and #10993.
Commit log:
```
ced4a27240 Include concurrency info in output (#10970)
a0b6800a93 Limit maximum number of concurrent tests (#10969)
a0fe0f6239 release 1.18.0-snapshot.20210921.7889.1.1b473c2b (#10993)
58c82b6e41 LF: reduce usage of NodeCreate.coinst (#10988)
f56ce2a757 kvutils: Add structured error data to the error metadata [KVL-1032] (#10964)
3f4dbf6f8e interfaces: Syntax highlighting for interfaces (#10987)
d52a2857da interface: Add test for conversion functions (#10991)
28b8d9a1f7 bump dotnet (#10979)
fe10ffbe8f Use ValueEnricher in ScenarioRunner. (#10897)
bdad7d5c99 Upgrade Java on Windows (#10965)
34390f7062 interfaces: Implemented conversions, added test. (#10982)
7740b60836 participant-integration-api: Improve the CommandConfiguration docs. (#10978)
04f322e2aa Bump resources for daml-ledger tests (#10984)
6d9922e1f1 Retry upsert of command deduplication on oracle and h2 [DPP-609] (#10976)
eb4f1b2a05 LF: Move lookup tests out of EngineTest (#10973)
99836d2610 Handle fetchByKey callback correctly in scenario runner (#10980)
23e6a2dddb Improve feedback from failing match in AuthMatchers (#10981)
e299103450 [Short] Small test tool's CLI parser refactor (#10923)
fc153a91b2 release 1.18.0-snapshot.20210921.7889.0.1b473c2b (#10971)
1b473c2b9a Switch trigger service time provider default to wall-clock time (#10967)
4acf34b5fc Add conversions from/to interfaces in Daml parser (#10954)
d50df11f77 interface: Add to/from interface in scala ast (#10960)
9f88e0919f Drop jdk8 override (#10963)
e0587bcdd2 Add a bit of detail to RegisteredTemplates documentation, motivated by: (#10962)
cebe6ed482 Refactor safetyStep in the simplifier. (#10948)
ac192fc6db interfaces: Add to/from interface in Haskell typechecker (#10951)
4b4d7a3af7 Remove update normalizer which was too aggresive (#10925)
66e1098070 Add tests for the `CompletionFromTransaction` converter [KVL-1104] (#10885)
724e50d739 interfaces: Add to/from interface in Haskell AST (#10945)
9a8d55ad13 Change slack link to discuss wiki. Fixes#10946 (#10947)
8b3b033fad LF: Test preprocessor resuming (#10936)
2edfc062a0 ifaces: name collision, typecheck fetch/exercise (#10896)
5dc15c6911 LF: rename language Interface to PackageInterface (#10938)
054c6abb63 Upgrade canton to a more recent version (#10944)
b8533d50e9 [JSON-API] Production/HA documentation. (#10903)
a33176265a Clarify `CommandTracker` [KVL-1104] (#10943)
5244643d16 Changes to increase timeout and remove assertions around mock CommandSubmissionServiceImpl (#10942)
6cc42ee9f4 rotate release duty after 1.17.0-snapshot.20210915.7832.0.38227a8e (#10892)
88ef05e557 sandbox-classic: Only allow `--max-parallel-submissions` here. (#10941)
ac02dbdeb9 LF: Exhaustive test for valueTranslator. (#10927)
409c0b4f60 interfaces: Add to/from_interface in proto (#10937)
02c8a9dfb8 Split CommonStorageBackend (#10871)
dc71a6aea8 update NOTICES file (#10932)
0ba54a4bf9 Add a missing test case to the `CommandTrackerFlowTest` (#10939)
855ecdf1a7 [DPP-572] Add ledger API test case for verifying conformance to `--min-tls-version` flag. (#10898)
3e13e3d87e Switch to stable urls in scoop python manifest (#10933)
fc2c87d950 ledger-api-client: Generate a submission id if it's empty in the `CommandClient` [KVL-1104] (#10926)
04d8f753b2 Clean up sandbox-on-x conformance tests. (#10766)
b4541b5a4d Logging delay of submitted commands (#10912)
b8e21d8f48 Fix takeFilter for the test StreamConsumer (#10918)
906368d7e2 LF: exhaustive test for CommandPreprocessor (#10914)
61d214e451 Add fetch, exercise implementations for interfaces in speedy. (#10911)
d01f8e1c35 Fix flaky ApiConfigManagementServiceSpec test (#10922)
50291ed61b interfaces: scala typechecker implementation (#10867)
cac8391d4c Bump ghc-lib to include daml interfaces parser (#10747)
b6a6bf7aac [Ledger API error codes] Extracted common errors and groups [DPP-607] (#10890)
308f938512 Dpp 494 unit testing ha coordinator (#10862)
30f74adc99 Mark Extractor's VeryLargeArchiveSpec test as flaky (#10916)
9582e019ec LF: Refactor PreprocessorSpec test (#10909)
9b0fa29aec Separate exercise & fetch for interfaces from templates (#10908)
f4adee91ca Add conformance test for command deduplication using the CommandService [KVL-1099] (#10883)
8a391189a6 Rename Completion.deduplication_time to deduplication_duration [KVL-1057] (#10900)
8e22bb6b2d Drop ContractId typeparameter from Value (#10827)
45437059d4 Drop cocreature as a codeowner from runtime things (#10906)
e6e81470bb Release 1.17.0 RC from second to last commit (#10904)
f08ac5f65c Desugar interface implements declarations (#10895)
b5648c0e3d Make `CommandTracker` distinguish submissions of the same command using `submissionId` [KVL-1104] (#10868)
```
Changelog:
```
[Triggers Service] The service now starts by default using wall-clock time instead
of static time. If you want to run using static time, you need to do so explicitly
using the new '-s' or '--static-time' CLI option. If you were already using '-w'
or '--wall-clock-time' the flag has no effect. It's anyway safe to leave it there.
Sandbox: Add CLI flag `--min-tls-version` to select minimum enabled TLS version for participant server.
```
CHANGELOG_BEGIN
CHANGELOG_END
The `main` branch build for #10971 failed. #10979 should have fixed it,
so let's try again.
This needs to be merged _after_ #10979.
CHANGELOG_BEGIN
CHANGELOG_END
