Commit Graph

8479 Commits

Author SHA1 Message Date
akshayshirahatti-da
e474b2d15a
[JSON-API] Websockets fix for matchedQueries (#11361)
* Changes to ensure matchedQueries are returned correctly when queries contain a mix of offsets and no offsets.

CHANGELOG_BEGIN
[JSON-API] fixes a bug related to the matchedQueries value returned for websocket multiqueries,
this only happens for patterns where the multiqueries contain a mixture of queries with and without
offsets.
CHANGELOG_END

* changes based on code review comments
2021-10-26 23:27:27 +00:00
fabiotudone-da
4a34b68500
KV: port V2 errors to self-service errors framework [KVL-1143] (#11326)
* KV: port V2 errors to self-service errors framework

CHANGELOG_BEGIN
CHANGELOG_END

* Fix ConversionsSpec compiling

* Use a valid ID for ValidationFailure

* Fix error details in Conversions

* Relax and simplify checks in KeyValueConsumptionSpec

* Fix formatting

* Fix definite answer in deduplication v2

* Try and make Scala 2.12 happy

* Fix typos in comments

* Use NOT_FOUND for unknown parties and provide them as resources

* Race (group) -> SubmissionRaces

* Don't deprecate resource exhausted (ABORTED) error

* Rename KVCompletionErrors group into KVErrors (ResourceExhausted can be synchronous)

* Remove unneeded braces

Co-authored-by: Hubert Slojewski <hubert.slojewski@digitalasset.com>

* Move toJsonString to where it's used

* Deprecate InvalidLedgerTime

* Embed deprecation comments into annotation

* Change deprecation messages so that they make more sense in the docs

* Move deprecated errors to a dedicated group as proposed by @tudor-da

Co-authored-by: Hubert Slojewski <hubert.slojewski@digitalasset.com>
2021-10-26 21:17:18 +00:00
tudor-da
811a6d3ed6
Fixed AuthorizationInterceptorSpec again (#11418)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 22:33:02 +02:00
Remy
c8006b866a
ScenarioRunner: enrich incomplete transactions (#11384)
* ScenarioRunner: enrich incomplete transactions

fixes #11352

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 19:28:05 +02:00
Kamil Bozek
d9c7031fc3
ACS testing - payload support [DPP-661] (#11308)
* Generating random payload of configurable size in ledger-api-bench-tool command submission

CHANGELOG_BEGIN
- [Integration Kit] - ledger-api-bench-tool can generate test contracts with configurable payload size.
CHANGELOG_END

* Fix for Scala 2.12

* Multi-template support for command submission in the ledger-api-bench-tool [DPP-659] (#11365)

* Added multi-template support for command submission in the ledger-api-bench-tool

CHANGELOG_BEGIN
- [Integration Kit] - Added multi-template support for command submission in the ledger-api-bench-tool
CHANGELOG_END

* Simplified distribution calculation

* Improved throttling - set max 100 in-flight commands
2021-10-26 17:19:03 +02:00
Stephen Compall
d87d3d490d
deal with deadlocks while fetching contracts in json-api Oracle (#11391)
* clean up some imports

* test case trying to find deadlock situation

* add deadlocks to causes of ContractsFetch retry for Oracle

* Revert "test case trying to find deadlock situation"

This reverts commit 9b19046b18.

* no changelog

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 11:03:50 -04:00
Hubert Slojewski
8212c0b2a5
Make submission ID optional [KVL-1107] (#11011)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 14:39:56 +00:00
Robert Autenrieth
3587eb84a2
Use Timestamp instead of Instant (#11356)
changelog_begin
changelog_end
2021-10-26 16:30:10 +02:00
Samir Talwar
ea5f09e524
sandbox: Deprecate the --eager-package-loading flag. (#11404)
It never did anything in Sandbox Next. Packages are always loaded
eagerly in KV ledgers.

CHANGELOG_BEGIN
- [Sandbox] The ``--eager-package-loading`` flag has been deprecated. It
  hasn't actually done anything for many releases; packages are always
  loaded eagerly. This does not affect Daml on SQL, which does support
  lazy package loading.
CHANGELOG_END
2021-10-26 16:10:11 +02:00
Brian Healey
9f882f2161
remove search index on json fields that harm insert and pruning performance (#11041)
* remove search index on json fields that harms insert performance

* Drop indexes on json array fields in participant_events_xxx tables

CHANGELOG_BEGIN
CHANGELOG_END

* Revert "remove search index on json fields that harms insert performance"

This reverts commit 78969f58
2021-10-26 09:21:54 -04:00
Brian Healey
70b90f47c1
optimize max event_sequential_id query for oracle (#11297)
* Tweak max event_sequential_id query for oracle to ensure consistent performance on larger tables

CHANGELOG_BEGIN
CHANGELOG_END

* respond to review feedback

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 09:20:39 -04:00
Moritz Kiefer
b1fed31ca4
Fix missing script results (#11395)
fixes #11007

changelog_begin

- [Daml Studio] Fix a bug where script results in Daml Studio
  sometimes do not show up.

changelog_end
2021-10-26 15:00:55 +02:00
nickchapman-da
03db0aa9f3
Auto run/check security evidence generation in ./fmt.sh (#11407)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 13:58:35 +01:00
pbatko-da
c928f0e560
[Short] Typo (#11400)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 12:28:43 +02:00
tudor-da
ba6c2be1b5
Add missing TransactionId to com.daml.error.ErrorResource (#11396)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 09:32:15 +00:00
nickchapman-da
a2a15716b4
Generate security evidence by documenting security testcases (#11306)
* Generate security evidence by documenting security testcases.

CHANGELOG_BEGIN
CHANGELOG_END

* move generated file to root of repo, so links work

* formatError function instead of Show instance

* dont use Show instances for generating Markdown

* magic comment: SECURITY_TEST --> TEST_EVIDENCE

* use megaparsec and Data.Text

* remove redundant T.pack

* use: Text.Megaparsec.Char.space
2021-10-26 09:07:59 +00:00
Moritz Kiefer
8d17882951
Allocate parties sequentially in script export tests (#11389)
* Allocate parties sequentially in script export tests

We’ve seen a few timeouts so this seems at least worth a try.

changelog_begin
changelog_end

* Extend logging to ease debugging

changelog_begin
changelog_end
2021-10-26 08:21:40 +00:00
Robert Autenrieth
1309c2f819
DPP-587 Use Timestamp instead of Instant (#11183)
* Use Timestamp instead of Instant

Time has microsecond resolution in Daml

changelog_begin
changelog_end

* Revert accidental newline changes

* Remove unused conversion modes

* Apply review comments
2021-10-26 08:54:47 +02:00
Moritz Kiefer
82f987390e
Rotate release rotation (#11394)
We forgot to do that last week because the automation didn’t create
the PR.

changelog_begin
changelog_end
2021-10-26 06:43:18 +00:00
tudor-da
b14077a607
Fix AuthorizationInterceptorSpec flake (#11387)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 21:18:37 +02:00
azure-pipelines[bot]
7090f2df9a
update NOTICES file (#11367)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-10-25 19:52:55 +02:00
Hubert Slojewski
ad42dfa915
Update gRPC to the latest (1.41.0) and Protobuf (#11380)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 17:26:55 +00:00
carrielaben-da
54c400a341
Update wording in Deploying to a generic Daml ledger (#11327)
* Update generic_ledger.rst

* Changelog

Additional wording changes for grammar and style.

CHANGELOG_BEGIN

CHANGELOG_END

Co-authored-by: carrie-laben <91496516+carrie-laben@users.noreply.github.com>
2021-10-25 17:12:25 +00:00
Robert Autenrieth
4461ed1561
Fix log output (#11374)
changelog_begin
changelog_end
2021-10-25 17:26:42 +02:00
Moisés Ackerman
613aac3e47
Add support for non-star-kinded type synonyms in data-dependencies (#11293)
* Add DA.Internal.NatSyn as a stable package

This module exposes a type NatSyn, to be used for encoding
type synonyms of Nat-kinded types

* Add generalized roundtrip test helpers in LFConversion tests

roundtripTestsBy and roundtripTestsPartialBy are like their
non-By versions, except they take an explicit equality predicate
instead of relying on the Eq instance. This allows the source
and target types to differ.

* Define encoder/decoder for type synonyms

This works by saturating the RHS of the declaration with artificial
variable names and adding the corresponding parameters on the LHS
In the case of Nat type synonyms, the Proxy-like
DA.Internal.NatSyn.NatSyn type is used to wrap the value into
something of kind star.

* Use type synonym encoder/decoder when generating/consuming DALF

This closes #11226

changelog_begin
changelog_end

* Extend type synonyms over data-dependencies test

* Add test cases for multi-param type class synonyms
2021-10-25 17:10:05 +02:00
Robin Krom
f89ecc6b95
interfaces: add an experimental toTypeRep builtin. (#11378)
This adds an experimental `toTypeRep: forall t. t -> TypeRep` builtin.
It will only work on interface payloads and crash horribly otherwise.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:09:23 +00:00
Gary Verhaegen
5654d5cb48
fix es ingest for missing files (#11375)
If a job fails at build time, there are not test logs to process.
Currently this means the ingestion process is going to be stuck in an
endless loop of retrying that job and failing on the missing file.

This change should let us process jobs with no test files.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:40:33 +02:00
tudor-da
03cfd1237c
Configurable assertions in Ledger API test tool by feature descriptors (#11328)
* ApiVersionService propagates self-service error codes flag.
* ParticipantTestContext is enriched with feature descriptors
* ContractIdIT adapted with assertions for self-service error codes

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:13:44 +02:00
pbatko-da
96b7b5812f
[DPP-648][Self-service error codes] Adopt ApiPartyManagementService (#11338)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 12:09:54 +00:00
Remy
9e94ae0f46
LF: move repl exception-auth test from dev to stable (#11369)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 11:13:43 +00:00
Remy
5365d68a38
LF: Remove PartialTransaction out from ScenarioRunner/IdeLedgerClient (#11368)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 13:07:23 +02:00
pbatko-da
79037c8855
[DPP-646][Self-service error codes] Adopt ApiPackageManagementService (#11314)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 12:11:15 +02:00
Kamil Bozek
0ee59f57c9
Command submission in the ledger-api-bench-tool. (#11296)
CHANGELOG_BEGIN
- [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes.
CHANGELOG_END
2021-10-25 09:57:53 +02:00
Remy
8d5cab5f16
LF: Simplify seeds generation in scenario runnner (#11353)
We just need the new seeds to be:
- fresh during a run,
- deterministic across two runs.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 08:54:42 +02:00
Moritz Kiefer
9e5b78804e
Speedup daml repl integration tests (#11335)
We’ve seen the tests time out despite already having a 900s test
timeout so this PR speeds up the tests using the following approaches:

1. Drop redundant test. We don’t need a test to test that daml repl
starts without a ledger if other tests also use that. Admittedly that
could make debugging slightly worse but I’m happy to accept that here.
2. Stop using a Ledger where we don’t need one. The import tests are
about testing the client side not the server side.
3. Share ledgers where we can. The inbound messages size tests and the
static time tests can reuse the ledger.
4. Stop using packages where we don’t need them. This speeds up both
ledger startup as well as Daml Repl startup.

changelog_begin
changelog_end
2021-10-25 08:52:38 +02:00
Stephen Compall
3bc0db3316
fix contract_tpid_fkey-related race condition (#11330)
* trying to reliably reproduce the template ID constraint error

* tentative fix for template ID constraint error

* sequential simulation

* successfully reproduce the error pre-4633c3137a

- Batch entry 0
        INSERT INTO some_fancy_prefix_contract
        VALUES ('foo', 1, 'null'::jsonb, NULL, '{}'::jsonb, ?, ?, '')
        ON CONFLICT (contract_id) DO NOTHING
       was aborted: ERROR: insert or update on table "some_fancy_prefix_contract" violates foreign key constraint "some_fancy_prefix_contract_tpid_fkey"
  Detail: Key (tpid)=(1) is not present in table "some_fancy_prefix_template_id".

* also reproduced the error pre-4633c3137a on Oracle

- ORA-02291: integrity constraint (UNA3GOHUV7YMSKT0MQXJKLKD9HKKAZ.SYS_C007859)
  violated - parent key not found

* add changelog

CHANGELOG_BEGIN
- [JSON API] Fixed a rare error that manifested as
  ‘violates foreign key constraint "contract_tpid_fkey"
   Detail: Key (tpid)=(...) is not present in table’
  when attempting to run queries and goes away on JSON API restart.
  See `issue #11330 <https://github.com/digital-asset/daml/pull/11330>`__.
CHANGELOG_END

* clean up some now-unneeded printlns
2021-10-22 11:46:26 -04:00
Gerolf Seitz
ab8a863734
[docs] Add Daml Driver for VMBC to the commercial integrations section (#11360)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:44:11 +00:00
Andreas Herrmann
c95db72e6c
Fix Bazel cache download retry (#11238)
* Stress test

* Fix Bazel cache download retry

changelog_begin
changelog_end

* Revert "Stress test"

This reverts commit b62f16c75a.

* Fix indentation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-10-22 15:34:43 +00:00
pbatko-da
e8d0ccbdb8
[DPP-611][Self-service error codes] Adapt ApiCommandService (#11325)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:42:38 +02:00
pbatko-da
a89079b4a5
[DPP-647][Self-service error codes] Adopt ApiParticipantPruningService (#11324)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:39:58 +02:00
tudor-da
cc8ec28481
[Self-service error codes] Adapt GrpcHealthService (#11354)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 13:37:36 +02:00
pbatko-da
c60c94b13f
[DPP-645][Self-service error codes] Adapt ApiConfigManagementService (#11312)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 12:28:15 +02:00
Moisés Ackerman
e6da1f7b52
Add step in ghc-lib guide for getting submodules to work (#11351)
changelog_begin
changelog_end
2021-10-21 15:55:44 +00:00
Robert Autenrieth
f3057ea03b
Increase timeout for non-repudation tests on Postgres (#11340)
changelog_begin
changelog_end
2021-10-21 16:27:32 +02:00
Robin Krom
176f470b25
interface: adding interfaces to the TS codegen (#11280)
* interface: adding interfaces to the TS codegen

We add TS interfaces corresponding to Daml interfaces to the generated
TS code.

CHANGELOG_BEGIN
CHANGELOG_END

* Update language-support/ts/codegen/README.md

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update language-support/ts/codegen/src/TsCodeGenMain.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* comment on pure interface methods

* added comment on substitutions

* extend example/tests to two implementations

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-10-21 14:22:46 +00:00
Robert Autenrieth
355352f2d1
DPP-650 Remove the mutating schema (#11211)
* Remove the mutating schema

changelog_begin
- [Participant] All participants now use the new append-only schema. Existing databases will
  automatically upgrade to the new schema the first time a participant/ledger is started.
changelog_end

* Fix post-commit validation test

* Remove append-only flags from CI

* Don't crash when using deprecated flag

* Increase timeout for reset service tests

* Fix typos in parameter names

* Restore removed test

* Restore removed CLI check

* Improve CLI parameter description
2021-10-21 14:40:35 +02:00
pbatko-da
443b64dffc
[DPP-621][Self-service error codes] Adopt error codes in ApiVersionService (#11302)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 14:03:10 +02:00
Sofia Faro
ed9dbed100
interfaces: Add fixed choice collision check in typechecker (Haskell) (#11337)
* interfaces: Add fixed choice name collision check

Add a check that a template cannot have two choices with the same name,
even taking into account all of its "inherited" interface fixed choices.

Part of #11137

changelog_begin
changelog_end

* "Me want" -> "We want"
2021-10-21 12:29:05 +01:00
pbatko-da
c37ecd1a42
[Short] Pass correct loggingContext to withValidatedPackageId (#11307)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 09:27:55 +00:00
pbatko-da
0d305cf212
[Short] Move field before logging statement (ApiTimeService) (#11313)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 09:46:34 +02:00