Commit Graph

8166 Commits

Author SHA1 Message Date
pbatko-da
c928f0e560
[Short] Typo (#11400)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 12:28:43 +02:00
tudor-da
ba6c2be1b5
Add missing TransactionId to com.daml.error.ErrorResource (#11396)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-26 09:32:15 +00:00
nickchapman-da
a2a15716b4
Generate security evidence by documenting security testcases (#11306)
* Generate security evidence by documenting security testcases.

CHANGELOG_BEGIN
CHANGELOG_END

* move generated file to root of repo, so links work

* formatError function instead of Show instance

* dont use Show instances for generating Markdown

* magic comment: SECURITY_TEST --> TEST_EVIDENCE

* use megaparsec and Data.Text

* remove redundant T.pack

* use: Text.Megaparsec.Char.space
2021-10-26 09:07:59 +00:00
Moritz Kiefer
8d17882951
Allocate parties sequentially in script export tests (#11389)
* Allocate parties sequentially in script export tests

We’ve seen a few timeouts so this seems at least worth a try.

changelog_begin
changelog_end

* Extend logging to ease debugging

changelog_begin
changelog_end
2021-10-26 08:21:40 +00:00
Robert Autenrieth
1309c2f819
DPP-587 Use Timestamp instead of Instant (#11183)
* Use Timestamp instead of Instant

Time has microsecond resolution in Daml

changelog_begin
changelog_end

* Revert accidental newline changes

* Remove unused conversion modes

* Apply review comments
2021-10-26 08:54:47 +02:00
Moritz Kiefer
82f987390e
Rotate release rotation (#11394)
We forgot to do that last week because the automation didn’t create
the PR.

changelog_begin
changelog_end
2021-10-26 06:43:18 +00:00
tudor-da
b14077a607
Fix AuthorizationInterceptorSpec flake (#11387)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 21:18:37 +02:00
azure-pipelines[bot]
7090f2df9a
update NOTICES file (#11367)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-10-25 19:52:55 +02:00
Hubert Slojewski
ad42dfa915
Update gRPC to the latest (1.41.0) and Protobuf (#11380)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 17:26:55 +00:00
carrielaben-da
54c400a341
Update wording in Deploying to a generic Daml ledger (#11327)
* Update generic_ledger.rst

* Changelog

Additional wording changes for grammar and style.

CHANGELOG_BEGIN

CHANGELOG_END

Co-authored-by: carrie-laben <91496516+carrie-laben@users.noreply.github.com>
2021-10-25 17:12:25 +00:00
Robert Autenrieth
4461ed1561
Fix log output (#11374)
changelog_begin
changelog_end
2021-10-25 17:26:42 +02:00
Moisés Ackerman
613aac3e47
Add support for non-star-kinded type synonyms in data-dependencies (#11293)
* Add DA.Internal.NatSyn as a stable package

This module exposes a type NatSyn, to be used for encoding
type synonyms of Nat-kinded types

* Add generalized roundtrip test helpers in LFConversion tests

roundtripTestsBy and roundtripTestsPartialBy are like their
non-By versions, except they take an explicit equality predicate
instead of relying on the Eq instance. This allows the source
and target types to differ.

* Define encoder/decoder for type synonyms

This works by saturating the RHS of the declaration with artificial
variable names and adding the corresponding parameters on the LHS
In the case of Nat type synonyms, the Proxy-like
DA.Internal.NatSyn.NatSyn type is used to wrap the value into
something of kind star.

* Use type synonym encoder/decoder when generating/consuming DALF

This closes #11226

changelog_begin
changelog_end

* Extend type synonyms over data-dependencies test

* Add test cases for multi-param type class synonyms
2021-10-25 17:10:05 +02:00
Robin Krom
f89ecc6b95
interfaces: add an experimental toTypeRep builtin. (#11378)
This adds an experimental `toTypeRep: forall t. t -> TypeRep` builtin.
It will only work on interface payloads and crash horribly otherwise.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:09:23 +00:00
Gary Verhaegen
5654d5cb48
fix es ingest for missing files (#11375)
If a job fails at build time, there are not test logs to process.
Currently this means the ingestion process is going to be stuck in an
endless loop of retrying that job and failing on the missing file.

This change should let us process jobs with no test files.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:40:33 +02:00
tudor-da
03cfd1237c
Configurable assertions in Ledger API test tool by feature descriptors (#11328)
* ApiVersionService propagates self-service error codes flag.
* ParticipantTestContext is enriched with feature descriptors
* ContractIdIT adapted with assertions for self-service error codes

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 14:13:44 +02:00
pbatko-da
96b7b5812f
[DPP-648][Self-service error codes] Adopt ApiPartyManagementService (#11338)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 12:09:54 +00:00
Remy
9e94ae0f46
LF: move repl exception-auth test from dev to stable (#11369)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 11:13:43 +00:00
Remy
5365d68a38
LF: Remove PartialTransaction out from ScenarioRunner/IdeLedgerClient (#11368)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 13:07:23 +02:00
pbatko-da
79037c8855
[DPP-646][Self-service error codes] Adopt ApiPackageManagementService (#11314)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 12:11:15 +02:00
Kamil Bozek
0ee59f57c9
Command submission in the ledger-api-bench-tool. (#11296)
CHANGELOG_BEGIN
- [Integration Kit] - The ledger-api-bench-tool is now capable of generating test contracts for testing purposes.
CHANGELOG_END
2021-10-25 09:57:53 +02:00
Remy
8d5cab5f16
LF: Simplify seeds generation in scenario runnner (#11353)
We just need the new seeds to be:
- fresh during a run,
- deterministic across two runs.

CHANGELOG_BEGIN
CHANGELOG_END
2021-10-25 08:54:42 +02:00
Moritz Kiefer
9e5b78804e
Speedup daml repl integration tests (#11335)
We’ve seen the tests time out despite already having a 900s test
timeout so this PR speeds up the tests using the following approaches:

1. Drop redundant test. We don’t need a test to test that daml repl
starts without a ledger if other tests also use that. Admittedly that
could make debugging slightly worse but I’m happy to accept that here.
2. Stop using a Ledger where we don’t need one. The import tests are
about testing the client side not the server side.
3. Share ledgers where we can. The inbound messages size tests and the
static time tests can reuse the ledger.
4. Stop using packages where we don’t need them. This speeds up both
ledger startup as well as Daml Repl startup.

changelog_begin
changelog_end
2021-10-25 08:52:38 +02:00
Stephen Compall
3bc0db3316
fix contract_tpid_fkey-related race condition (#11330)
* trying to reliably reproduce the template ID constraint error

* tentative fix for template ID constraint error

* sequential simulation

* successfully reproduce the error pre-4633c3137a

- Batch entry 0
        INSERT INTO some_fancy_prefix_contract
        VALUES ('foo', 1, 'null'::jsonb, NULL, '{}'::jsonb, ?, ?, '')
        ON CONFLICT (contract_id) DO NOTHING
       was aborted: ERROR: insert or update on table "some_fancy_prefix_contract" violates foreign key constraint "some_fancy_prefix_contract_tpid_fkey"
  Detail: Key (tpid)=(1) is not present in table "some_fancy_prefix_template_id".

* also reproduced the error pre-4633c3137a on Oracle

- ORA-02291: integrity constraint (UNA3GOHUV7YMSKT0MQXJKLKD9HKKAZ.SYS_C007859)
  violated - parent key not found

* add changelog

CHANGELOG_BEGIN
- [JSON API] Fixed a rare error that manifested as
  ‘violates foreign key constraint "contract_tpid_fkey"
   Detail: Key (tpid)=(...) is not present in table’
  when attempting to run queries and goes away on JSON API restart.
  See `issue #11330 <https://github.com/digital-asset/daml/pull/11330>`__.
CHANGELOG_END

* clean up some now-unneeded printlns
2021-10-22 11:46:26 -04:00
Gerolf Seitz
ab8a863734
[docs] Add Daml Driver for VMBC to the commercial integrations section (#11360)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:44:11 +00:00
Andreas Herrmann
c95db72e6c
Fix Bazel cache download retry (#11238)
* Stress test

* Fix Bazel cache download retry

changelog_begin
changelog_end

* Revert "Stress test"

This reverts commit b62f16c75a.

* Fix indentation

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-10-22 15:34:43 +00:00
pbatko-da
e8d0ccbdb8
[DPP-611][Self-service error codes] Adapt ApiCommandService (#11325)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:42:38 +02:00
pbatko-da
a89079b4a5
[DPP-647][Self-service error codes] Adopt ApiParticipantPruningService (#11324)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 15:39:58 +02:00
tudor-da
cc8ec28481
[Self-service error codes] Adapt GrpcHealthService (#11354)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 13:37:36 +02:00
pbatko-da
c60c94b13f
[DPP-645][Self-service error codes] Adapt ApiConfigManagementService (#11312)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-22 12:28:15 +02:00
Moisés Ackerman
e6da1f7b52
Add step in ghc-lib guide for getting submodules to work (#11351)
changelog_begin
changelog_end
2021-10-21 15:55:44 +00:00
Robert Autenrieth
f3057ea03b
Increase timeout for non-repudation tests on Postgres (#11340)
changelog_begin
changelog_end
2021-10-21 16:27:32 +02:00
Robin Krom
176f470b25
interface: adding interfaces to the TS codegen (#11280)
* interface: adding interfaces to the TS codegen

We add TS interfaces corresponding to Daml interfaces to the generated
TS code.

CHANGELOG_BEGIN
CHANGELOG_END

* Update language-support/ts/codegen/README.md

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update language-support/ts/codegen/src/TsCodeGenMain.hs

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* comment on pure interface methods

* added comment on substitutions

* extend example/tests to two implementations

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-10-21 14:22:46 +00:00
Robert Autenrieth
355352f2d1
DPP-650 Remove the mutating schema (#11211)
* Remove the mutating schema

changelog_begin
- [Participant] All participants now use the new append-only schema. Existing databases will
  automatically upgrade to the new schema the first time a participant/ledger is started.
changelog_end

* Fix post-commit validation test

* Remove append-only flags from CI

* Don't crash when using deprecated flag

* Increase timeout for reset service tests

* Fix typos in parameter names

* Restore removed test

* Restore removed CLI check

* Improve CLI parameter description
2021-10-21 14:40:35 +02:00
pbatko-da
443b64dffc
[DPP-621][Self-service error codes] Adopt error codes in ApiVersionService (#11302)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 14:03:10 +02:00
Sofia Faro
ed9dbed100
interfaces: Add fixed choice collision check in typechecker (Haskell) (#11337)
* interfaces: Add fixed choice name collision check

Add a check that a template cannot have two choices with the same name,
even taking into account all of its "inherited" interface fixed choices.

Part of #11137

changelog_begin
changelog_end

* "Me want" -> "We want"
2021-10-21 12:29:05 +01:00
pbatko-da
c37ecd1a42
[Short] Pass correct loggingContext to withValidatedPackageId (#11307)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 09:27:55 +00:00
pbatko-da
0d305cf212
[Short] Move field before logging statement (ApiTimeService) (#11313)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-21 09:46:34 +02:00
Robert Autenrieth
73c94b5b38
Increase timeout for non-repudation test (#11281)
changelog_begin
changelog_end
2021-10-21 08:01:29 +02:00
pbatko-da
88c607b781
[Self-service error codes] Adapt ApiTransactionService [DPP-613] (#11094)
* [DPP-417][DPP-613] Adopt self-service error codes ApiTransactionService

CHANGELOG_BEGIN
CHANGELOG_END

* TV review

* TV review

* Uses ErrorFactories

* Error loggers use enriched logging contexts

* Deleted ApiTransactionServiceSpec

* Add TODO wrt to CommandValidation.invalidArgument

* Adapt transactionNotFound error assertions in conformance tests

* ErrorFactories.invalidArgumentWasNotFound for handling invalid event ids

* Revert improvement to TransactionNotFound cause message since
Canton still outputs the old message and assertions fail.

Co-authored-by: Tudor Voicu <tudor.voicu@digitalasset.com>
2021-10-20 22:18:26 +00:00
Moritz Kiefer
07ad3e0d9a
Suport multi-party readAs in triggers (#11299)
* Suport multi-party readAs in triggers

fixes #7640

This does not yet include the trigger service. We’ll tackle that separately.

changelog_begin

- [Daml Triggers] Triggers now support readAs parties. They can be
  specified via `--ledger-readas a,b,c`. As part of this change
  ``testRule`` gained an extra argument to specify the `readAs`
  parties. If you previously used

  ```
  testRule trigger party acsBuilder commandsInFlight s
  ```

  you now need to use

  ```
  testRule trigger party [] acsBuilder commandsInFlight s
  ```

changelog_end

* Update triggers/tests/src/test/scala/com/digitalasset/daml/lf/engine/trigger/test/AbstractFuncTests.scala

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>

Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
2021-10-20 18:17:15 +00:00
Sofia Faro
76eb1657d2
Interface fixed choices: ghc parser (#11275)
* Interface fixed choices: ghc parser

(WIP)

changelog_begin
changelog_end

* .

* ..

* ...

* stackage unpin unix

* ...

* fold create/observer/signatory into fake lfconversion primitives

* stackage unpin

* .....

* ......

* ........

* .........

* . . . . .

* unpin stackage on unix

* ...... ......

* ...

* . . . . .

* unpin stackage on unix

* . . . . . . . . .

* unpin stackage on unix

* Loosen restriction on ETo/FromAnyChoice fake primitives

* uncomment exercise in Interface.daml

* remove comments

* !!!

* the final re-pinning, part 1

* the final re-pinning: part 2
2021-10-20 17:45:13 +00:00
pbatko-da
da27a1e812
[DPP-619][Self-service error codes] Adopt error codes in ApiVersionService (#11303)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-20 16:38:22 +02:00
pbatko-da
5f5af30ce0
[DPP-628][Self-service error codes] Adapt error codes in ApiTimeService (#11295)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-20 15:37:01 +02:00
tudor-da
f9e67adafc
[Self-service error codes] Adapt error responses in ledger-api-auth [DPP-617] (#11223)
* [Self-service error codes] Implement V2 in Authorizer

CHANGELOG_BEGIN
CHANGELOG_END

* Added unit test for authorize (non-streamed)

* Fix after rebase

* Do not expose the error codes switching mechanism to the Java bindings

* Adjust InternalAuthorizationError to be SystemInternalAssumptionViolated

* Parameter names in test

* Testing AuthorizationInterceptor with regard to returned error codes

* Do not use default error code version switchers at instance creation

* Addressed Pawel's review comments

* Using ErrorFactories for error dispatching

* Pass loggingContext to Authorizer where available

* Generic internal authorization error
2021-10-20 13:28:21 +00:00
Moritz Kiefer
728296575b
Fix component status for triggers (#11311)
changelog_begin
changelog_end
2021-10-20 15:28:13 +02:00
Moritz Kiefer
17776f3496
Factor out npm install step of create-daml-app tests (#11294)
* Factor out npm install step of create-daml-app tests

This PR speeds up the create-daml-app tests by 30s or so by factoring
out the install of the unchanged npm deps into a genrule which we then
feed in again. This is a bit ugly but given how frequently we run
those tests I do think it is worth the uglyness.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-10-20 14:24:30 +02:00
pbatko-da
3a8b685346
[Short] Fix docs for Dispatcher#startingAt (#11304)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-20 14:02:04 +02:00
azure-pipelines[bot]
f315a9003b
release 1.18.0-snapshot.20211019.8113.0.8ff347d8 (#11300)
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@sofiafaro-da is in charge of this release.

Commit log:
```
8ff347d8ed Expand type synonyms lazily (#11282)
2fd200fff8 KV: support contextual logging in KeyValueConsumption [KVL-1143] (#11288)
f4df1cc4d0 converting server errors to proper client errors (#11184)
525e4ceb5e Add GrpcStatus.toProto(JavaStatus) -> ScalaStatus conversion [KVL-1143] (#11292)
9b00a1aab5 Rotate release rotation (#11291)
dd09c38f4a Upgrade rules-nodejs (#11290)
87f1418d91 [Short] Remove unnecessary traits from ApiSubmissionServiceSpec (#11254)
6b65a72577 Merge npm install steps in create-daml-app tests (#11287)
a9f6afbfde kvutils: Rename `VersionedOffset` to `KVOffset`. (#11286)
98cf8d86b3 KV: introduce v2 error codes behind a CLI switch [KVL-1140] (#11224)
46f6877ee5 Increase time model skew limits (#11273)
8f94cffdd1 kvutils: Use `VersionedOffsetBuilder` where possible, and introduce `VersionedOffsetMutator`. [KVL-1154] (#11277)
81fde97b87 Bazel: Call `_wrap_rule` directly when building the Scala REPL rule. (#11279)
edb2b04abf Document scaladoc usage in Bazel (#11278)
62558374a7 Data dependencies support for reexports (#11147)
2fc7490df2 [Self-service error codes] Adapt error factories [DPP-656] (#11270)
b1a6b11d47 ledger-api-test-tool: Add static time awareness [KVL-1156] (#11266)
243c120939 Add an LF typechecking benchmark (#11276)
2db8c138a8 Test authorization within Exercise choice. (#11246)
ec58ed68b3 Typecheck nested LF type applications more efficiently (#11253)
a940016f46 Add debugRaw corresponding to traceM in Haskell (#11259)
a9885797fe Fix Scala repl targets (#11272)
e4808f620e Extract UtilLF module from daml-lf-conversion to its own library (#11263)
38712c0179 sandbox-next - Disable participant command deduplication [KVL-1156] (#11264)
82fa229385 Add daml-util-ghc lib (#11260)
68a93cf70f kvutils - Use the ledger configured time for command dedup [KVL-1149] (#11239)
c8cede074e Bump startup time in create-daml-app tests (#11261)
4fac87f112 Bump the schema version for the JSON API (#11252)
55076700ae update NOTICES file (#11256)
0063b1037f Retry faster during startup (#11255)
04feb40a8f Improve reset service tests (#11240)
b3375fd2a1 update compat versions for 1.18.0-snapshot.20211013.8071.0.514e8b50 (#11237)
9ed1eb3ce3 Address security notice on `ansi-regex` (#11243)
387d0e8f6c Make compatibility migrations `eternal` (#11242)
be899b32a8 Extract rejection_reason.proto from daml_kvutils.proto [KVL-1090] (#11235)
a1d94e10ba LF: Create special case class for VersionedContractInstance (#11127)
b7389885e8 Release 1.18 snapshot (#11234)
3c26852226 LF: Use template Id in exercise node of fixed choice. (#11229)
139b6f3b9e CI: Set `PROJ_DIR` inside the bash lib, not outside. (#11236)
ce64cb2a13 interfaces: Do some TODOs (#11231)
7a88c7d97c trigger-service: dev-mode-unsafe flag (#11233)
909a1bf896 [DPP-417][DDP-612] Adapt ApiSubmissionService to support V2 error codes (#11052)
```
Changelog:
```
- [JSON API] Several kinds of gRPC server errors are now reported with
  associated HTTP statuses; for example, a Daml-LF interpreter error now
  returns a 400 instead of a 500, and an exercise on an archived contract
  returns a 409 Conflict instead of a 500.  Errors internal to JSON API
  (e.g. internal assertion failures) are no longer detailed in the HTTP
  response; their details are only logged.
  See `issue #11184 <https://github.com/digital-asset/daml/pull/11184>`__.

- [Daml Stdlib] Add `debugRaw` as a convenience wrapper around
  `traceRaw` when used inside a do-block. `debugRaw` compares to
  `debug` like `traceRaw` compares to `trace` meaning it expects a
  `Text` instead of calling `show` on an expression.

sandbox-next - Disable participant command deduplication

kvutils - Command deduplication uses the configured time model (static/wall) and not running always using wall-clock
[JSON API] Solving a bug that could cause the JSON API to return
correct result if a contract with the same key is observed twice
required a schema change. The JSON API data needs to be dropped
and the query store needs to reset. If you are migrating from a
previous version, either reset your database manually or start
the HTTP JSON API with one of the options that regenerate the
schema (`create-only`, `create-if-needed-and-start`, `create-and-start`).
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-10-20 12:58:01 +01:00
pbatko-da
8a3abce47b
[DPP-618][Self-service error codes] Adapt error codes in ApiPackageService (#11284)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-20 13:19:45 +02:00
Moritz Kiefer
50ea92f7b3
Use ApiTypes.Party instead of String in the trigger runner (#11298)
changelog_begin
changelog_end
2021-10-20 10:06:20 +02:00