Commit Graph

8450 Commits

Author SHA1 Message Date
Victor Peter Rouven Müller
7974427765
release 1.18.0-snapshot.20211117.8399.0.a05a40ae (#11740)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-17 15:26:17 +00:00
tudor-da
a05a40aeec
[Self-service error codes] Adapt ResetService in Sandbox classic (#11731)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-17 11:11:54 +01:00
Marton Nagy
eca53e9ede
Integrate ACSReader [DPP-720] (#11688)
* Integrating ACSReader and 2 phase retrieval logic to TransactionReader
* Add metrics and debug logging

changelog_begin
changelog_end
2021-11-17 09:46:24 +00:00
Stefano Baghino
bf4f9f7eea
Document ./release.sh new snapshot (#11730)
changelog_begin
changelog_end
2021-11-17 09:10:43 +01:00
pbatko-da
122a4ccd0d
[DPP-417][Self-service error codes][Docs] Mention that correlation id is usually submission id. (#11728)
CHANGELOG_BEGING
CHANGELOG_END
2021-11-17 08:07:44 +00:00
mziolekda
ccbf714078
expand the text about errors in the log [DPP-755] (#11723)
* expand the text about errors in the log

* scrap the entire logging section
2021-11-16 22:21:35 +00:00
tudor-da
fbc436d70a
[DPP-739][Self-service error codes] Adapt QueueBackedTracker error codes (#11719)
* [Self-service error codes] Adapt QueueBasedTracker

CHANGELOG_BEGIN
CHANGELOG_END

* Added changed error codes to migration guide

* Fixed ErrorFactoriesSpec

* Update ledger/error/src/main/scala/com/daml/error/definitions/LedgerApiErrors.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: Pawel Batko <pawel.batko@digitalasset.com>
Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-16 22:21:29 +00:00
pbatko-da
07cff7cdb7
[DPP-417][Self-service error-codes] Document error categories. (#11727)
* [Self-service error-codes] Docs intro WIP

* CHANGELOG_BEGIN
CHANGELOG_END

* 1

* Fix rst

CHANGELOG_BEGIN
CHANGELOG_END

* [DPP-417][Self-service error-codes] Document error categories.

CHANGELOG_BEGIN
CHANGELOG_END

* 1

* Fix
2021-11-16 22:55:35 +01:00
Marton Nagy
c866fa6701
Optimize data migrations for performance [DPP-714] (#11714)
* Refactoring interning data migrations for performance
* Planned improvement: migration-verification

changelog_begin
changelog_end
2021-11-16 19:43:22 +00:00
Moritz Kiefer
f801314c16
Drop exercise with actor from LF spec (#11721)
This doesn’t exist since LF 1.5 and we don’t support that or any LF
version before anymore.

changelog_begin
changelog_end
2021-11-16 18:31:36 +01:00
Miklos
a63e091068
kvutils: Use consistent argument name casing in KVErrors (#11609)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 17:30:29 +00:00
tudor-da
c1f4beb328
Ledger ID mismatch does not change error codes. (#11720)
* Remains NOT_FOUND

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 17:05:09 +00:00
Stephen Compall
b5b29fd38a
test_suite for http-json, http-json-oracle integration tests (#11713)
* make http-json:integration-tests into test suites

* make http-json-oracle:integration-tests into test suite

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END

* remove commented libraries
2021-11-16 11:52:26 -05:00
pbatko-da
a29fee925c
[Self-service error-codes] Add introductory section to self-service error codes page. (#11712) 2021-11-16 16:41:41 +01:00
Hubert Slojewski
639c4319e8
libs-scala: Move SourceQueueResourceOwner from the Integration Kit (#11708)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 14:59:19 +00:00
tudor-da
05f604d672
[Self-service error codes] Enrich migration guide [DPP-755] (#11711)
* Adapted migration guide to include changes from https://github.com/digital-asset/daml/pull/11686

CHANGELOG_BEGIN
CHANGELOG_END

* Enriched migration guide with Sandbox-classic changed gRPC error codes

* Added migration guide for error codes changed in KV committer

* Extract the KV and Sandbox-classic migration guides

* Hierarchical order of error codes docs

* Added back full grpc error code description to columns
* Using "gRPC status code" instead of "gRPC error code"
* Addressed review comments from Hubert

* Apply suggestions from code review

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>

* Addressed review comments
* Use 'Error codes' instead of 'Self-service error codes'

Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
2021-11-16 14:25:53 +00:00
Moritz Kiefer
2d5b54828b
Use explicit party specifications in Daml Script over JSON API (#11678)
* Use explicit party specifications in Daml Script over JSON API

This uses the new party set arguments from #11454 to allow Daml Script
over the JSON API to `submit p` even if we have a token with more
claims.

changelog_begin

- [Daml Script] When run over the JSON API, Daml Script can now use a
  subset of the claims in the token. E.g., `submit p` works even if you
  have a token with `actAs = [p, p2]`.

changelog_end

* Apply suggestions from code review

Co-authored-by: Remy <remy.haemmerle@daml.com>

* Drop opt prefix

changelog_begin
changelog_end

* switch around error & success cases

changelog_begin
changelog_end

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-11-16 12:17:32 +00:00
Samir Talwar
844e5d0420
participant-integration-api: Propagate log context through the indexer. (#11710)
And log the participant ID on startup for kvutils apps.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-16 11:16:51 +01:00
Oliver Seeliger
c9692666ae
Formatting feedback from: Log execution context throwables as error (#11707)
Follow-up from https://github.com/digital-asset/daml/pull/11702

changelog_begin
changelog_end
2021-11-16 09:45:22 +01:00
tudor-da
ec6d7cc3db
[Self-service error codes] Error code definitions revisiting [DPP-675] (#11686)
* Error definitions revisiting and hierarchical grouping refactoring

CHANGELOG_BEGIN
CHANGELOG_END

* Full hierarchical path for error grouping

* Addressed Pawel's review comments

* Apply suggestions from code review

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>

* Do not strip margin in error code explanations

* Revert changing LEDGER_ID_MISMATCH from NOT_FOUND to FAILED_PRECONDITION

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>
2021-11-15 21:28:13 +00:00
nickchapman-da
a1fc5c6a25
Simplify handling for EqualList builtin. (#11676)
Step 1: SEBuiltinRecursiveDefinition(_) -> SEBuiltinEqualList

CHANGELOG_BEGIN
CHANGELOG_END

Step2: Replace SEBuiltinEqualList with SBuiltin.SBEqualList

update comment
2021-11-15 17:49:07 +00:00
Kamil Bozek
90ad968d39
Workflow for the ledger-api-bench-tool defined in YAML file [DPP-669] (#11682)
* Moved StreamConfig to a separate class WorkflowConfig

* Change streams from Option[List[]] to List[]

* Simplifying

* Dedicated PartyFilter case class

* Use String instead of Identifier in filter config to allow using short names

* Submission config

* Switched to unified WorkflowConfig

* Minor change

* Minor change

* Workflow config parser tests

* Removed leftovers

* Added more tests

* Removed leftovers

* Updated CLI

* Complete stream configuration support in the YAML file in the ledger-api-bench-tool

CHANGELOG_BEGIN
- [Integration Kit] - ledger-api-bench-tool - All stream configuration parameters can be defined through the YAML configuration file.
CHANGELOG_END

* Addressed review comments (renaming).
2021-11-15 18:47:05 +01:00
akshayshirahatti-da
1791a522c9
[TS-Bindings] Changes to re-enable ws multiplexing (#11681)
* Changes to renable ws multiplexing

CHANGELOG_BEGIN
[TS-BINDINGS] Re-enable ws multiplexing for stream queries after resolving the reconnect connection close bug associated with ws state and liveness.
CHANGELOG_END

* websocket is passed as an argument to the onMessage handler

* consistently use 'manager' reference instead of 'this' in the handleQueries change method
2021-11-15 22:57:21 +05:30
azure-pipelines[bot]
fd940997ca
update compat versions for 1.18.0-snapshot.20211111.8349.0.d938a44c (#11667)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-15 17:32:01 +01:00
Kamil Bozek
612a838987
ledger-api-bench-tool: Allocate observers indexed from 0 until the total number (#11700)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-15 17:13:08 +01:00
Moritz Kiefer
41d2d950f4
Factorize Java codegen integration test build defs (#11704)
changelog_begin
changelog_end
2021-11-15 16:59:04 +01:00
Oliver Seeliger
bf86ee4f5c
Log execution context throwables as error (#11702)
* Log execution context throwables as error

changelog_begin
changelog_end

* Review feedback and reuse more logging contexts
2021-11-15 15:27:07 +00:00
Hubert Slojewski
69471d6135
ledger-api-client: Fix a log message related to an empty submission ID (#11695)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-15 13:57:19 +00:00
Remy
775b41c235
LF: Reintroduce TransactionVersion.asVersionedTransaction (#11692)
* LF: Reintroduce TransactionVersion.asVersionedTransaction

This have been drop in #11626, but canton is using it.

CHANGELOG_BEGIN
CHANGELOG_END

* cosmetic

* cosmetic
2021-11-15 12:07:23 +00:00
Stefano Baghino
8015f60708
Add cpu:2 tag to extractor tests (#11697)
This will limit the parallelism of extractor integration tests.

Each test runs now a bit faster although overall all tests seem
to run a bit slower. This seems a good compromise as it should
limit the contention over system resources between various
integration tests, each running a PostgreSQL instance and sharing
various system resources.\

changelog_begin
changelog_end
2021-11-15 11:39:05 +00:00
tudor-da
8e08450220
[Self-service error codes] No redundant logging in Ledger API (#11649)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-15 09:13:50 +01:00
Andreas Lochbihler
18433eb19e
fix arity of TArrow type constructor in Daml-LF spec (#11677)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-15 07:49:45 +01:00
Marton Nagy
fb840fa82f
Introduce filter table [DPP-714] (#11687)
* Adding new table, indexes
* Adding data migration (PostgreSQL only)
* Removing unnecessary indexes
* Adding DbDto, Schema integration for the filter table
* Adding generation of CreateFilter Dto-s to UpdateToDbDto
* Adding sequential ID generation to sandbox-classic and parallel-ingestion
* Adding StorageBackend support for pruning, reset
* Adding EventStorageBackend support for two phase retrival queries
* Extending/adapting unit tests
* Adding PostgreSQL VACUUM ANALYZE Flyway migration script as well (since this is the last persistence change in the ACS feature track)

changelog_begin
changelog_end
2021-11-14 23:12:40 +01:00
Marton Nagy
cb5a675cfb
Activate template_id interning [DPP-713] (#11680)
* Add DB schema changes
* Data migration scripts (PostgreSQL only)
* Adapt queries, result-parsers
* Adapt ingestion

changelog_begin
changelog_end
2021-11-12 23:49:27 +00:00
tudor-da
c4da8e327e
Limit gRPC wire transport size legacy error codes size (#11666)
* Limit gRPC wire transport size legacy error codes size

CHANGELOG_BEGIN
CHANGELOG_END

* Add detail around limiting the Status message size.

* Update ledger/ledger-api-common/src/main/scala/com/digitalasset/platform/server/api/validation/ErrorFactories.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

* Reformat

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-12 19:04:20 +00:00
Marton Nagy
72f3dd4090
Activate party interning [DPP-712] (#11663)
* Add DB schema changes
* Data migration scripts (PostgreSQL only)
* Add StringInterning to Query strategies
* Adapt queries, result-parsers
* Adapt ingestion
* Fix JdbcLedgerDaoBackend StringInterning handling

changelog_begin
changelog_end
2021-11-12 19:01:02 +00:00
Sofia Faro
57f1b868d3
Improve byInterfaceNodes based on suggestions (#11672)
* Improve byInterfaceNodes based on suggestions

From the last PR #11662

changelog_begin
changelog_end

* use transaction traversal order
2021-11-12 17:31:47 +00:00
tudor-da
d6ca68dc23
[Self-service error codes] Adapt Sandbox classic rejection error codes (#11622)
* [Self-service error codes] Adapt Sandbox classic rejection error codes
* Unify common rejection error codes with KV
* Adapt conformance tests

CHANGELOG_BEGIN
CHANGELOG_END

* Fix JdbcLedgerDaoPostCommitValidationSpec

* Addressed first round of review comments

* Addressed Miklos' comments

* Revert camelCasing invalidLedgerTime arguments

* Moves DuplicateCommand to CommandRejections

* Addressed Gerolf's comments
2021-11-12 16:33:02 +00:00
Remy
15b925a116
LF: Compute transaction version as the maximum of the children (#11626)
fixes #11377

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 17:00:07 +01:00
azure-pipelines[bot]
ebdab313e9
rotate release duty after 1.18.0-snapshot.20211109.8328.0.92181161 (#11624)
@remyhaemmerle-da is taking care of 1.18.0-snapshot.20211109.8328.0.92181161 (#11623), so they get pushed back to the end of the line.

Please do not merge this before #11623.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-12 16:52:46 +01:00
Remy
7eb22ddf4f
LF: drop forgotten deprecate Node aliases. (#11673)
Because a bug in the scala compiler deprecated pattern matching are
not detected (See https://github.com/scala/bug/issues/12493).
Hence some deprecated usage of Node aliases have been forgotten in

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 15:07:22 +00:00
Moritz Kiefer
b0463aaab2
Move script auth tests to postgresql (#11670)
changelog_begin
changelog_end
2021-11-12 13:55:25 +01:00
nickchapman-da
8c46559163
Remove unnecessary constructors: SEDamlException, SEImportValue, from SExpr{0,1} (#11668)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 12:40:52 +00:00
Moritz Kiefer
c109552a6e
Disable ofInterestRule by default (#11637)
This seems like a more sensible default. There are really only very
few places where it makes sense to do things automagically (mainly the
IDE). For everything else we might just end up wasting time by
compiling stuff we don’t need.

changelog_begin
changelog_end
2021-11-12 11:01:58 +01:00
Sofia Faro
409864f220
Use by_interface field when fetching transaction dependencies. (#11662)
* Use by_interface field when fetching tx deps.

Fixes #10915

changelog_begin
changelog_end

* scalafmt
2021-11-12 09:37:24 +00:00
pbatko-da
81e24504d4
[DDP-596][Self-service error codes] Add migration guide (#11632)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-12 10:07:37 +01:00
nickchapman-da
3192d5eb74
Different types before after closure conversion (#11661)
* avoid single letter (s./t.) import prefixes

* Split type SExpr0 -> SExpr{0,1} for before/after closure conversion

CHANGELOG_BEGIN
CHANGELOG_END

* remove unnecessary constructors in SExpr{0,1}

* remove SExpr0.SExprAtomic
2021-11-12 08:54:43 +00:00
tudor-da
2f4476c12a
[Self-service error codes] Fill in missing places with submission id for correlation id (#11593)
* [Self-service error codes] Fill in missing places with submission id for correlation id
* KVConsumption
* GrpcCommandSubmissionService

CHANGELOG_BEGIN
CHANGELOG_END

* Addressed review comments

* WriteService uses submission id as correlation id

* Populate the submission id in command submissions in ledger-api-test-tool

* Do not fail submission on missing submission id

* Fix KeyValueParticipantStateWriterSpec

* Addressed review comments
2021-11-12 09:08:06 +01:00
Moritz Kiefer
d3bb036591
Fix sandbox database config in script & trigger tests (#11655)
* Fix sandbox database config in script & trigger tests

Setting jdbcUrl does absolutely nothing because it later gets
overwritten again.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-11-12 06:18:12 +00:00
Stephen Compall
22d916baed
party-set arguments for JSON API (#11454)
* add actAs, readAs to `meta` for create, exercise, createAndExercise endpoints

* use meta actAs, readAs to control how contract IDs are looked up for exercise

* outdated comments on JwtWritePayload and JwtPayload

* toSet1 operator to clean up some NEL manipulation

* take optional readAs argument for query endpoint

* use readAs for query POST

* check whether readAs is safe in query endpoint

* missed CommandMeta args in tests

* FetchRequest, a domain model to layer on "fetch" endpoint's ContractLocator

- ContractLocator was overloaded as a domain request model *and* a component
  of other domain request models; the addition of new arguments means it can
  no longer exactly meet the former, and adding "readAs" to it would poison it
  for the latter cases.

* take readAs argument from fetch endpoint

* add readAs security check from query to fetch

* move jwt parties functions to util

* testing the party-set/JWT functions

* missing headers

* caught boolean blindness in readAs security checks

* test that meta params are used for commands

* make resolveRefParties do a subset check, too

* Revert "make resolveRefParties do a subset check, too"

This reverts commit 40a66f102c.

* test that the readAs auth check actually applies

* test that command service uses meta readAs, actAs

* note on test coverage

* add changelog

CHANGELOG_BEGIN
- [JSON API] ``actAs`` and ``readAs`` may be specified for create, exercise,
  create-and-exercise, non-WS fetch, and non-WS query.
  See `issue #11454 <https://github.com/digital-asset/daml/pull/11454>`__.
CHANGELOG_END

* no saving mallocs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* untabify

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* move design comment to comment from function name

- suggested by @cocreature; thanks

* remove unneeded variable

* refactor single-key callers of requestJsonReader

- suggested by @cocreature; thanks

* build error in ce

* diagnose Windows failure

* add missed http-json-testing requirement

* use readers as fetch/query party-set name

- suggested by @cocreature and @realvictorprm, thanks

* extra import

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2021-11-12 02:10:15 +00:00