Commit Graph

8544 Commits

Author SHA1 Message Date
Sofia Faro
2f8f69e7cb
Drop DA.Next.Set and DA.Next.Map (#11864)
* Drop DA.Next.Set and DA.Next.Map

Fixes #11527. Also removed a bunch of unnecessary CPP, though there's
still a lot more CPP to remove.

changelog_begin

- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
  after being deprecated since Daml-LF 1.11

changelog_end

* Drop a deriving MapKey

* update unstable-types test
2021-11-25 13:02:08 +00:00
pbatko-da
5f3a4d2067
[Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867)
Also make section header markers consistent docs' reamde.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 13:46:12 +01:00
tudor-da
cf3ac011ca
[Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828)
* [Self-service error codes] Do not return error code id in metadata

CHANGELOG_BEGIN
CHANGELOG_END

* Do not propagate definite_answer
* Additionally, remove redundant context info from error codes

* Fix tests

* Create IndexDbException as a specialization
* Used to globally define error codes returned by the persistence layer
that are logging
2021-11-25 12:42:29 +01:00
Simon Meier
026b92a8b1
Add gRPC definitions for participant user management service (#11818)
* Introduce definitions for experimental user management service.

Only gRPC definitions are added. No new service is exposed by the Ledger
API server.

CHANGELOG_BEGIN

- [Ledger API] Introduce gRPC definitions for experimental user
  managament service to manage users and their rights for interacting
  with the Ledger API served by a participant node.

CHANGELOG_END
2021-11-25 12:26:47 +01:00
Andreas Herrmann
2fde30d916
Disable writing volatile bits in Scala statsfile (#11875)
Make Scala targets deterministic by disabling writing milliseconds
timestamps into the statsfile.

Before (cache disabled)
```
$ n=1; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ n=2; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ diff -u execlog1.json execlog2.json
--- execlog1.json       2021-11-25 09:53:45.638891575 +0100
+++ execlog2.json       2021-11-25 10:01:52.831225558 +0100
@@ -311392,7 +311392,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile",
     "digest": {
-      "hash": "1c535baa94812c16d33b6b3dcf2dec8e12b77e3a28cd1280ca2c9678840c4de6",
+      "hash": "a7f6f1b168649c1c5346ff92c69f0095b68781690e4b3f33321d8f35af8b3d77",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
@@ -311705,7 +311705,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile",
     "digest": {
-      "hash": "b3af662512a906ec56abfb222f43af189b88b00ae0449bd44c25b7e90689c4db",
+      "hash": "d6c6e2a36754efea4863593a838fbfb500a66e0150d3c3e2e2490ad65c22a2cd",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
@@ -312025,7 +312025,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/libs-scala/adjustable-clock/adjustable-clock.statsfile",
     "digest": {
-      "hash": "4326088227e5ecf2c21e76ddb6d77c9692ef1689dd146f65ec1e75c235382251",
+      "hash": "8a6b23f3c64ce3ff171772808ca0064af52bc145a32d1b6398de124aeb813c97",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
$ diff -r output1 output2
diff -ur output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile
--- output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile      2021-11-25 09:53:41.706824833 +0100
+++ output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile      2021-11-25 10:01:49.047160467 +0100
@@ -1 +1 @@
-build_time=2507
+build_time=2522
diff -ur output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile
--- output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile    2021-11-25 09:53:39.110780774 +0100
+++ output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile    2021-11-25 10:01:46.427115402 +0100
@@ -1 +1 @@
-build_time=4162
+build_time=3946
diff -ur output1/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile output2/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile
--- output1/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile    2021-11-25 09:53:45.282885532 +0100
+++ output2/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile    2021-11-25 10:01:52.511220053 +0100
@@ -1 +1 @@
-build_time=3158
+build_time=3100
```

After (cache disabled)
```
$ n=3; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ n=4; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ diff -u execlog3.json execlog4.json
$ diff -ur output3 output4
```

See https://github.com/bazelbuild/rules_scala/pull/1298

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-25 11:01:17 +00:00
Andreas Herrmann
4ed9deda39
Remove xxd from dev-env (#11876)
The corresponding Nix attribute is missing
```
$ xxd
[dev-env] Building tools.xxd...
error: attribute 'xxd' in selection path 'tools.xxd' not found
[dev-env] Build of tools.xxd has failed!
```

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-25 11:01:04 +00:00
Gary Verhaegen
eaded41aca
remove mergify (#11866)
It's been working pretty well for us, but our usage is now completely
covered by GitHub's auto-merge feature, which has the advantage of not
requiring us to duplicate our branch protection rules.

This will also reduce our security audit surface as that's one fewer
provider to scan (not that we've found anything wrong with them so far,
but still, less work in the future).

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 11:36:24 +01:00
Andreas Lochbihler
3cd5028b61
fix a few more things in the daml-lf spec (#11851)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2021-11-25 10:46:19 +01:00
Marton Nagy
beca0ee3e9
Refactor StandaloneApiServer factory (#11842)
To enable direct IndexServer building (and enable two step ApiServer build process)
This will be needed for KV WriteService proxy: a wrapper is being build to serve deduplication time conversion, which needs access to completion service.

changelog_begin
changelog_end
2021-11-25 10:12:23 +01:00
Hubert Slojewski
6356f13cde
Properly upgrade gRPC to 1.41.0 (#11858)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 08:50:33 +00:00
tudor-da
f6accd3a2f
Release 1.18 RC2 (#11869)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 08:36:05 +01:00
Gary Verhaegen
d858873c44
fix main (#11868)
sorry!

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 23:11:59 +00:00
azure-pipelines[bot]
da8dd7ee2f
rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845)
@nickchapman-da is taking care of 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11844), so they get pushed back to the end of the line.

Please do not merge this before #11844.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-24 19:35:46 +00:00
pbatko-da
066da4f30c
[Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 19:23:06 +01:00
Stefano Baghino
258fb65d1c
Document how to deal with HTTP JSON API schema changes (#11336)
* Document how to deal with HTTP JSON API schema changes

changelog_begin
[HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes
changelog_end

Furthermore: uses HTTP JSON API instead of HTTP-JSON API consistently,
drops the capitalization of query store in the documentation and fixes
hyphening in a few places

* Fix title markup
2021-11-24 18:05:30 +00:00
Gary Verhaegen
b8937ad81b
ci: self-service compat test start (#11853)
It should now be possible for anyone to trigger the daily compat build
by adding a `run-full-compat: true` trailer to their commit message.

CHANGELOG_BEGIN
CHANGELOG_END

run-full-compat: true
2021-11-24 18:05:04 +00:00
Gary Verhaegen
de8d15fb1e
fix Nix install on macOS nodes (#11696)
As part of the 2.4 release, the Nix installer has been changed to take
care of the volume setup (which we don't want it to do here). Because
that requires root access, they've decided to make multi-user install
the default, and to disable single-user install.

We could do an in-depth review of the difference and adapt our setup to
use a multi-user setup (we do use the multi-user setup on Linux, so
there's precedent), but as an immediate fix, we can keep our single-user
setup _and_ get the latest Nix by using the 2.3.16 installer and then
upgrading from within Nix itself. This _should_ keep working at least
for a while, as Linux still defaults to single-user.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 18:53:13 +01:00
Stefano Baghino
b3d1d40b50
Expose submissionId via the Java bindings (#11839) (#11847)
* Expose submissionId via the Java bindings

Closes #11705

Will need to be forward-ported to `main` once merged.

changelog_begin
[Java Bindings] submissionId is now exposed via the bindings, see issue #11705
changelog_end

* Address https://github.com/digital-asset/daml/pull/11839#discussion_r755445100
2021-11-24 17:38:56 +01:00
Remy
86da6e8eef
LF: Test scala interface type checking (#11833)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 15:58:45 +00:00
Gary Verhaegen
5f52f00afb
increase linux cluster size (#11860)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 15:18:56 +00:00
Sofia Faro
5c12d757f4
Add a guard when exercising by interface. (#11836)
* Add a guard when exercising by interface.

This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.

It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.

I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)

changelog_begin
changelog_end

* scalafmt

* Feedback and fix matches

* Update comments, we are always going to abort the transaction

* Raise WronglyTypedContract in SBGuardTemplateId.

* rebase and fix parser

* restore ANF

* scalafmt
2021-11-24 14:22:40 +00:00
fabiotudone-da
7c3a2a7801
Add a new KV submission failure error (#11854)
CHANGELOG_BEGIN
[Integration Kit] Add a new SUBMISSION_FAILED internal error
CHANGELOG_END
2021-11-24 13:10:36 +00:00
Robert Autenrieth
aebc5a734f
All packages must be valid (#11850)
changelog_begin
changelog_end
2021-11-24 13:10:27 +00:00
nickchapman-da
0374843143
speedy compilation benchmark (#11852)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 13:10:18 +00:00
Remy
393893a601
LF encoder: make package validation optional (#11849)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 13:53:52 +01:00
Robert Autenrieth
25b476fdb2
DPP-726 Add string interning unit tests (#11841)
* DPP-726 Add string interning unit tests

changelog_begin
changelog_end

* Add more test cases
2021-11-24 13:38:38 +01:00
nicu-da
59eb0d2eff
kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848)
changelog_begin
kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`.
changelog_end
2021-11-24 04:32:34 -08:00
nickchapman-da
970243dd46
Ensure stack-safety during closure-conversion. (#11778)
ClosureConversion -> Suffix with "Old"

CHANGELOG_BEGIN
CHANGELOG_END

ClosureConversion old-vs-new diff check

ClosureConversionNew, first cut. All tests in SBuiltinTest work.

In addition we change some Array --> List in SExpr1 (for human pp).
And we throw away ClosureConversionDup.

adapt AnfTest from Array to List change for SExpr1

all tests pass in daml-lf/interpreter

remove SExpr0.SELet1General

reorder things

testing for stack-safety of closure conversion

file/class renames

improve naming

pass cont as sep arg to commit (move out of Up/Down)

comment stack-safe closure conversion

fix bug: failed to use env1

fix 2x unmoored doc comment

comment stack safety testing

Remove old closure-conversion code & diff-check between old/new.

loose StackSafe suffix on ClosureConversion class/file

rename StackSafetyTest.scala to ClosureConversionTest.scala

prefer "sealed abstract class" to "sealed trait"

fvs.zipWithIndex --> fvs.view.zipWithIndex

(SExpr1) SEAppGeneral -> SEApp; prefer List to Array in SEApp/SECase

prefer xs.toArray to Array(xs: _*)

access SExpr0 via "source."

two more .view

improve comment and fix typo

link to Issue

switch to a continuation stack; avoids nesting in the Cont type
2021-11-24 11:47:39 +00:00
nickchapman-da
e63c80dddd
update LATEST (#11846)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 11:18:34 +00:00
Hubert Slojewski
db425216d1
libs-scala: Change SourceQueueResourceOwner to BoundedSourceQueueResourceOwner [KVL-1177] (#11832)
CHANGELOG_BEGIN

- [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on

CHANGELOG_END
2021-11-24 12:02:19 +01:00
Hubert Slojewski
109b60613c
Make the InstrumentedSource.queue use the BoundedSourceQueue [KVL-1177] (#11807)
CHANGELOG_BEGIN

- [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future

CHANGELOG_END
2021-11-24 11:26:39 +01:00
Robert Autenrieth
bd2a6852ee
Make ACS reader parameters configurable (#11732)
changelog_begin
changelog_end
2021-11-23 21:48:02 +01:00
tudor-da
329e609ec4
[Self-service error codes] Group explanations (#11715)
* [Self-service error codes] Error group descriptions

CHANGELOG_BEGIN
CHANGELOG_END

* Apply suggestions from code review

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2021-11-23 20:30:33 +00:00
Moritz Kiefer
32e9c4b13a
Bump go rules (#11829)
* Bump go rules

Was debugging something else and this seemed simple enough to factor
it out.

changelog_begin
changelog_end

* Bump Bazel on Windows

changelog_begin
changelog_end

* bump bazel manifest

changelog_begin
changelog_end
2021-11-23 18:58:55 +00:00
azure-pipelines[bot]
6df7af7544
update compat versions for 1.18.0-snapshot.20211117.8399.0.a05a40ae (#11743)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-23 18:13:57 +01:00
pbatko-da
9461702309
[DPP-673][Self-service error codes] Split one narrow migration table into multiple narrower ones to fit in the generated pdf's pages. (#11803)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 17:38:31 +01:00
pbatko-da
55191847cd
[DPP-673][Self-service error-codes] Generate error code directory compatible with both pdf and html docs (#11794)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 15:30:48 +01:00
Moritz Kiefer
5a2c070da4
Skip append-only migration tests on each PR (#11816)
fixes #11760

changelog_begin
changelog_end
2021-11-23 15:00:50 +01:00
Stephen Compall
e6c8229594
port #11798 documentation changes to json-api HTTP codes documentation (#11823)
- see c4b6a17da1

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 13:16:46 +00:00
Remy
1610d977dc
LF: parsing interface primitives (#11825)
Continue work start in #11797

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-23 12:03:03 +00:00
mziolekda
f2aa09c114
Allow encoded server key files to be in base64 [DPP-761] (#11796)
* Allow encoded server key files to be in base64

CHANGELOG_BEGIN
CHANGELOG_END

* format

* dry tests
2021-11-23 09:49:52 +00:00
azure-pipelines[bot]
cc3f551030
update NOTICES file (#11824)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-23 08:30:31 +01:00
tudor-da
1bfcbc978b
Refactoring for Canton compatibility on PackageServiceErrors (#11812)
* Refactoring for Canton compatibility on PackageServiceErrors
* ErrorResource.all is public for accessibility from Canton

CHANGELOG_BEGIN
CHANGELOG_END

* Keep PackageServiceError object
2021-11-22 19:14:03 +00:00
tudor-da
c4b6a17da1
[Self-service error codes] Update existing error code references [DPP-593] (#11798)
* Updated Ledger API services protobuf definitions

CHANGELOG_BEGIN
CHANGELOG_END

* Adapted Building Applications / Ledger API / gRPC section

* Moved Error Codes under Building Applications / Ledger API / gRPC

* Small rewording Error codes page

* Updated outdated LedgerAPI / gRPC page

* Apply suggestions from code review

Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>

* Moved Error Codes under Building Applications / Ledger API / gRPC

* Updated outdated LedgerAPI / gRPC page

* Addressed review comments

* Addressed review comments

* Remove unnecessary link to error-codes

* `Ledger API` changed to `Common Ledger API change`

Co-authored-by: pbatko-da <pawel.batko@digitalasset.com>
2021-11-22 18:39:25 +00:00
Moritz Kiefer
addb2edc03
Drop Scala 2.13 version guards from Bazel definitions (#11819)
changelog_begin
changelog_end
2021-11-22 17:37:49 +00:00
Moritz Kiefer
5785bbfcfd
Drop LF < 1.14 from supported damlc output versions (#11701)
* Drop LF < 1.14 from supported damlc output versions

fixes #11319

We keep test coverage by depending on the most recent snapshot which
still has 1.14 support.

changelog_begin

- [Daml Compiler] Damlc can only produce Daml-LF 1.14 or
  newer. Passing aynthing older to `--target` is an error. If you
  need to produce older versions, use an older SDK.

changelog_end

* Switch around legacy_compiler_lf_versions

changelog_begin
changelog_end

* drop since-lf

changelog_begin
changelog_end
2021-11-22 16:20:17 +00:00
Remy
ae8391ee01
LF Parser: handle interface (#11797)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-22 14:49:02 +01:00
nicu-da
faddba4222
Increase range for excluded snapshots in compatilbility tests (#11802)
changelog_begin
changelog_end
2021-11-22 05:43:53 -08:00
Sofia Faro
f8933d1f4b
Make lookupTemplateChoice return only choices in the template. (#11808)
This fixes a bug in the typechecker (#11558) and the command
preprocessor, since those were written with this behavior of
lookupTemplateChoice in mind. Enables the engine test that
caught this.

changelog_begin
changelog_end
2021-11-22 13:27:48 +00:00
azure-pipelines[bot]
3e0ac71e99
update NOTICES file (#11801)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-22 12:29:39 +00:00