This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
Commit log:
```
84e1f3a7f1 participant-integration-api: Move transaction requests to structured logging. [KVL-996] (#10141)
b79d02e28e `ledger-api-bench-tool` - exposed metrics for Prometheus [DPP-471] (#10103)
b61e519a3a Release ledger API test tool per LF version (#10142)
2dfe026cc2 add ES cluster (#10144)
ef9a04caf5 Divulgence crash tests [DPP-433] (#9942)
bb4641756f Daml assistant capitalization (#10140)
a6ee10be9a Move visibility checks into speedy. (#10136)
c764fbe573 contextualized-logging: Introduce different logging types for more structure. [KVL-996] (#10134)
678cab0327 10050 append only schema on oracle (#10051)
f5e5062470 ensure that signatories and observers are disjoint in ledger-api CreatedEvents (#10123)
ffc88d52b4 print version of the ledger api test tool as part of the report (#10119)
621cfa4a91 Resolve contract keys conflicts in disclosed contracts (#9948) (#10034)
f1ffd52e08 Java bindings: add DamlRecord, deprecate Record (#10132)
3df2566305 Scenario: move Scenario Error Throwable to scenario package (#10075)
7521ec9988 [JSON-API] Include contract id in the logging ctx when logging exercise commands (#10131)
e0e333377c update NOTICES file (#10129)
398300b76f LF: Move Speedy Interpretation Error to transaction package (#10091)
d4150ac078 Refactor error reporting in Daml Repl (#10118)
0108fedd4e update NOTICES file (#10112)
6af36fe1e3 Fix Tx.Metadata for normalized transactions (#10108)
7745693648 Remove duplicate index (#10084)
1b617ae22d DPP-428 Add missing indices for index initialization (#10083)
af9382ce92 contextualized-logging: Reduce the API surface and avoid name collisions. (#10102)
01d6770450 json-api perf tests: combine large-ACS frequencies combinatorially instead of in lockstep (#10101)
a44afcff42 Upgrade nixpkgs (#9908)
9498d1509b [DOCS] Let jwt.io link to an already filled in working payload (#10026)
4affb053e9 Force newer version of glob-parent in Navigator (#10105)
f745f10394 LF: Realease LF 1.14 (#10077)
05056ddd8c Cut a flaky expectation some more slack (#10106)
46a66e2f2a Upgrade jest to the latest version (#10107)
84e329885f update compat versions for 1.11.2 (#10104)
a6b536f3c4 Compiler: Make LF 1.13 the default output (#9907)
29ddc88dcb remove redundant spaces in the log format (#10093)
01e329f2de DPP-432 Add exception tests to the JdbcLedgerDao suite (#10040)
3d79cbf2b1 Bump compat versions (#10096)
5bf7d8f570 Fix speedy perf reporting (#10100)
0970821ce7 Bump cpu alloc for build-and-lint (#10099)
cbca779657 [In-memory fan-out] Ensure getTransactionLogUpdates with max fetch size (#10064)
c756153951 Release SDK 1.11.2 (#10094)
3a4235400d rotate release duty after 1.15.0-snapshot.20210622.7213.0.d867d904 (#10086)
ecc2d11536 recommend some daml compiler warnings to enable (#10082)
b2bb45e4d9 Bump daml repl timeout (#10089)
db60d15b5e Log ledger api validation failures at info level (#10080)
f0dc025ac9 Release 1.15 snapshot (#10090)
```
Changelog:
```
- [Ledger API Server] The amount of data logged in the API transaction
service has been reduced at INFO level. Please enable TRACE logging to
log the request data structures.
- [Integration Kit] - ledger-api-bench-tool - exposed metrics via Dropwizard metrics (e.g. for Prometheus)
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
In case a contract key is already present in a past contract in the contract table nullify it. This gets rid of contract keys of previously disclosed contracts. We never discover that disclosed contracts get archived, so we get conflicts on such past keys
[Java bindings] In order to avoid clashing with `java.lang.Record` (introduced
in Java 14), `com.daml.ledger.javaapi.data.Record` has been renamed to
`com.daml.ledger.javaapi.data.DamlRecord`. The old name has been used to
denote a sub-type of the newly renamed one, so it can still be used, but it has
been marked as deprecated.
[Java codegen] The Java codegen now uses the `DamlRecord` type wherever `Record`
was used before.
- [JSON-API] The contract id is now also included in the logging ctx when logging exercise commands
- [LF] Add support for Excepction
- [Compiler] Default ouput LF version is now 1.13
Log ledger api validation failures at info level
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* participant-integration-api: Remove the subscription ID.
Pretty sure it's not used except for a single log line, which makes it
useless as a correlation ID.
If we want to correlate logs, let's add a correlation ID.
* participant-integration-api: Move transaction requests to trace logging.
Most of the useful information is already in the logging context. We
don't need to log the data structure too.
CHANGELOG_BEGIN
- [Ledger API Server] The amount of data logged in the API transaction
service has been reduced at INFO level. Please enable TRACE logging to
log the request data structures.
CHANGELOG_END
* participant-integration-api: Reorder methods in ApiTransactionService.
* participant-integration-api: Add the word "request" to some log lines.
* participant-integration-api: Add a logging prefix for string offsets.
* ledger-api-domain: `immutable.Set` -> `Set`.
It's an alias.
* participant-integration-api: Log transaction filters on subscription.
* participant-integration-api: Log transaction filters.
Just the parties isn't enough information.
* participant-integration-api: Log the entire transaction request.
Structured, because otherwise it's hard to throw things away later.
* contextualized-logging: Avoid `View` because it's not in Scala 2.12.
* contextualized-logging: Add tests for booleans.
* contextualized-logging: Avoid methods that accept views.
Scala 2.12 really doesn't like me when I do that.
* participant-integration-api: One more try at building with Scala 2.12.
Currently it is impossible to test new features until they land in the
default LF version. This is clearly not great. This PR releases one
Ledger API test tool per LF version (with the LF version being in the
name) to make it easier for Canton and others to test new features.
Note that at least the way things are setup now we won’t go back in
time. We will only publish stable, preview & dev but not older
versions. If needed, we could expand that in the future.
changelog_begin
changelog_end
This PR adds a basic ES cluster to our infrastructure, completely open
and unprotected but only accessible through VPN.
And, as of yet, through its IP address. I'm not sure whether it's worth
adding a DNS for it.
CHANGELOG_BEGIN
CHANGELOG_END
* A test case for crashing participant in a multi-node env with divulged contracts
* Fixed test naming
CHANGELOG_BEGIN
CHANGELOG_END
* Additional usability check for the disclosed contract
* A newline
* Verify that fetching an archived contract is not possible
* Second test case
* A `ledger-api-test-tool` test case for divulging a contract with key twice on multiple participants [DPP-433] (#9970)
* A test case for divulging a contract with a key twice on multiple participants
CHANGELOG_BEGIN
CHANGELOG_END
* Use autogenerated archive method
* Trigger build
* Increased timeout for TransactionServiceIT:TXInvisibleTransactionTreeByEventId
changelog_begin
changelog_end
- DAML -> daml
- Assistant -> assistant
The capitalization of the latter fixed one occurrence of a capitalized 'A'
which was inconsistent with the vast majority of uncapitalized 'a's.
* contextualized-logging: Automatically convert logging values to strings.
For now, this has almost the same behavior, but it allows us to
customize the output in the future.
The main change is that the log format has gone from:
context: {a=b, x=1, foo=bar}
to:
context: {a: "b", x: "1", foo: "bar"}
* contextualized-logging: Move `writeTo` inside `LoggingValue`.
* contextualized-logging: Allow for more than just strings.
`null`, numbers, and sequences are now correctly logged.
The log format has gone from:
context: {a: "b", x: "1", foo: "bar", parties: "[alice, bob]"}
to:
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_BEGIN
- The log output of Daml components has changed so that the structured
part is closer to JSON. This allows us to distinguish and parse
numbers and lists. If you are parsing this log output, you may need to
change your parser.
The log output has changed from:
.. code-block::
context: {a=b, x=1, foo=bar, parties=[alice, bob]}
to:
.. code-block::
context: {a: "b", x: 1, foo: "bar", parties: ["alice", "bob"]}
CHANGELOG_END
* contextualized-logging: Extract the string serializer.
* Use non-string logging where possible.
* contextualized-logging: Split logging values from serialization.
So that callers don't have to know about Jackson.
* contextualized-logging: `SeqView` is `Iterable`. Don't need both.
* contextualized-logging: Make `ToStringToLoggingValue` a `val`.
Co-Authored-By: Stephen Compall <stephen.compall@daml.com>
* contextualized-logging: Add a transient dependency for 2.12 only.
This required more infrastructure than I thought it would.
* kvutils: Make it explicit that we're logging the hashes of archives.
The implicit was found to be a little confusing.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Oracle compliant append only schema
CHANGELOG_BEGIN
CHANGELOG_END
WIP : oracle on new appendonly schema
* diff to postgres dump, create consolidated view
* diff to postgres dump to ensure all oracle setup is equiv
* recompute sha for changed oracle flyway scripts
* drop old tables to prevent clash on name of new participant_events view
* recompute sha for flyway script
* prelim oracle StorageBackend
* Adds support for special preparedStatement for oracle
* Add support and wires to setObject by default everywhere
* Add the full OracleField suite with TODOs for convenience
* Wires OracleField suite to OracleFieldStrategy
changelog_begin
changelog_end
* enable debug version of oracle driver
* conversion Instant -> Timestamp for oracle
* WIP: primitive println debugging
* Passing PackagesSpec with appendonlyschema on Oracle
Rename size column to siz to avoid reserved word clash, including migration script for postgres
* include sha for new postgres migration script
* add missing copyright header
* cleanup
* passing party spec for appendonly on oracle
* passing configuration spec for appendonly on oracle
* scalafmt
* bazel buildifier reformat
* use db generic FETCH NEXT n ROW ONLY rather than limit for cross db compat
* siz instead of size for packages table on all dbs and schema
* revert enabling oracle jdbc debug
* Support Array[String] -> String conversion (and vice versa) for JSON array
Remove as aliases for tables as this does not work with oracle
Extract submitters clause for all db types
Use append transaction injector for oracle append only spec
* scalafmt
* correct oracle failing active contract spec tests
* wire in JdbcLedgerDaoCompletionsSpec
* remove semi-colons for ending statements that are problematic for oracle driver
* all tests up to divulgence passing for append only on oracle
* all appendonly tests passing on oracle
* remove ignore on fall back to limit-based query with consistent results
* do not change name of size column in packages table for mutable schema for all DBs
* do not change name of size column in packages table for mutable schema for all DBs
* standalone oracle appendonly schema script
regen shas on flyway scripts
revert some cosmetic refactoring in CommonStorageBackend
* Fixes conversion to parties from Oracle-JSON at flatEventWitnessesColumn
* Switches from composit SQLs to single SQLs at prepared statements to accommodate Oracle limitation
* Fixes arrayIntersectionWhereClause by applying patch from mutable Oracle schema integration
* Fixes queries with empty startExclusive Offsets
* First draw adding Oracle conformance test suites to CI
* wire in the oracle conformance tests for CI
* Use cross-db fetch next $n rows only syntax instead of limit syntax that works only for postgres/h2
* rename siz to package_size
* recompute shas
* scalafmt and include sha check for oracle append only flyway script
* correct missing package_size rename
* remove some todos -- correct corrupted V1__Init.sql
* Update ledger/ledger-on-sql/src/test/lib/scala/com/daml/ledger/on/sql/MainWithEphemeralOracleUser.scala
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* correct version number for postgres rename column scripts
* remove unnecessary migration tables for oracle append only
* review feedback: rename createEventFilter as requested, remove todos
* review feedback: case consistency
* review feedback: update todos with issue markers
* review feedback: cleanup
* review feedback: OracleField and OracleSchema cleanup
* Fixing Table generators to use preparedData for convenience
* Placing TODOs for refactorings later
* Renames initial append-only oracle script, for convenience
* Falls back to original behavior as far prepared statements go at couple of queries
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* Resolve contract keys conflicts in disclosed contracts (#9948)
* build queries for nullifications of past key
* format code
CHANGELOG_BEGIN
In case a contract key is already present in a past contract in the contract table nullify it. This gets rid of contract keys of previously disclosed contracts. We never discover that disclosed contracts get archived, so we get conflicts on such past keys
CHANGELOG_END
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* correct oracle implemantation (new)
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Java bindings: add DamlRecord, deprecate Record
Fixes#10130
changelog_begin
[Java bindings] In order to avoid clashing with `java.lang.Record` (introduced
in Java 14), `com.daml.ledger.javaapi.data.Record` has been renamed to
`com.daml.ledger.javaapi.data.DamlRecord`. The old name has been used to
denote a sub-type of the newly renamed one, so it can still be used, but it has
been marked as deprecated.
[Java codegen] The Java codegen now uses the `DamlRecord` type wherever `Record`
was used before.
changelog_end
Boy-scout rule:
- removed references to ~old~ ancient versioning
- used `@deprecated` Javadoc annotation wherever meaningful
- some import re-arrangement performed by the linter
* Address https://github.com/digital-asset/daml/pull/10132#discussion_r659705929
* Refactor error reporting in Daml Repl
fixes#10098
As mentioned in that issue, the current behavior is a mess which
silently drops errors in certain cases. This PR switches things around
so that errors are reported to the client and we print them there
making sure that no error should ever get lost.
changelog_begin
changelog_end
* Bump timeout
changelog_begin
changelog_end
* Add missing indices for index initialization
changelog_begin
changelog_end
* Do not create unnecessary indices
* Create indices in H2
* Fix Oracle migration hash
* Verify all migration hashes
In #10016, 1% template ID and 1% party-set membership meant _the same_ 1%,
meaning that an index of both couldn't possibly yield interesting results. This
changes how LargeAcs builds the large ACS so that it's "1% of 1%", as you'd
expect.
CHANGELOG_BEGIN
CHANGELOG_END
This addresses a dependabot alert.
No nice way to do this unfortunately, it’s pulled in via transitive
deps which don’t have a version with less restrictive bounds.
I did test manually that Navigator still works (it’s a dev-time only
dependency).
changelog_begin
changelog_end
This ends up transitively bumping yargs-parser which has a
vulnerability but more generally I also just like being on the latest version.
changelog_begin
changelog_end
* Add exception tests
changelog_begin
changelog_end
* Add another test
This one tests a transaction that can only be
produced by a privacy-aware ledger.
* Address review comments
@S11001001 is taking care of 1.15.0-snapshot.20210622.7213.0.d867d904 (#10085), so they get pushed back to the end of the line.
Please do not merge this before #10085.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* CHANGELOG_BEGIN
Log ledger api validation failures at info level
CHANGELOG_END
* log validation failures in Api*Service family of classes
* address review comments
* WIP: reinterpret must fail
changelog_begin
changelog_end
EXPLORE: add x/y prefix to reinterpret
MiniEngineTest.scala
MiniTests.daml
setup reinterpret testing one step down, to be able to see rollback nodes
MiniEngineTest.scala - refactor yreinterpret
first minimal reinterpret test
setup testcase for reinterpret on a command which throws
catch exception during reinterpret; and test
introduce type for Validating; avoid boolean blindness
remove exploratory MiniTests
remove exploratory x/y prefix on reinterpret
question
fix build
* small cleanup
* add `unsafeCompileForReinterpretation`; preserve interface of existing `unsafeCompile` (command list)
* remove the Validating type (not needed for this work)
* undo format change
* more private
* update comment for Engine.reinterpret
* test ensure failure works correctly; no rollback expected
* refactor to share duplicated code in reinterpret tests
* remove unhelpful create/ensure tests
* test that reinterpretation of an uncatchable error remains a failure
* test that rollback-version-check-error remains a failure for reinterpretation
* remove dev comment left in by accident
* Introduce shell.nix to expose the Nix tooling, if required.
* dev-env: Rewrite the ghcide script to use nix-shell.
* language-support/hs: Use `nix-shell` for export-package.sh.
* Nix: Add a shebang to the Bazel wrapper script.
CHANGELOG_BEGIN
CHANGELOG_END
* dev-env: Standardize the `set` header in `ghcide`.
* dev-env: Replace `gpg` and `make` with symlinks.
* dev-env: Remove sphinx-autobuild. It's brought in by pipenv.
* dev-env: Remove Python 3.6 references. They seem to be unused.
CHANGELOG_BEGIN
CHANGELOG_END
Since per-request offset can be described strictly as a special case of the new
per-query offset semantics, go ahead and describe it that way, so that really
only one model needs to be understood to fully comprehend the query request
semantics.
CHANGELOG_BEGIN
CHANGELOG_END
