Commit Graph

8519 Commits

Author SHA1 Message Date
Gary Verhaegen
4e50060021
self-service compat: set branch name to not main (#11902)
The "tell slack about failures" step is conditional on the branch name
being main; for whatever reason if we don't supply a branch name Azure
sets it to `main`, regardless of the fact that the commit is not in that
branch at all.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 16:50:53 +00:00
nickchapman-da
2f4aa4797e
refactor to avoid impossible code path (#11901)
changelog_begin
changelog_end
2021-11-26 15:46:26 +00:00
Gary Verhaegen
a81995c82e
switch dev images to Temurin (#11895)
The AdoptOpenJDK project has been renamed to Adoptium so they renamed
the adoptopenjdk images to Temurin.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 14:45:35 +00:00
Moritz Kiefer
f3a0e2eba1
Set scalafmt dialect explicitly (#11898)
Scalafmt started issuing warnings because we don’t do this since our
latest nixpkgs upgrade which included a scalafmt upgrade.

changelog_begin
changelog_end
2021-11-26 13:57:57 +00:00
mziolekda
60e372dcd2
Don't run pruning tests on H2, they time-out (#11897)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 13:06:03 +00:00
Remy
58e69ade1a
LF: replace "dev" LF version by "1.dev" in bazel files (#11894)
the more consistent, as asked by Moritz in review of #11820

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 12:23:17 +00:00
nickchapman-da
8ef348dab0
Use absolute stack locations in SExpr1 (#11877)
* SExpr1.SELocS - carry relBad/absGood - abs unused so far

* compute/pass SELocS.abs in ClosureConversion, and check in Anf that it matches the reconstructed value

CHANGELOG_BEGIN
CHANGELOG_END

* Remove relative stack locations. Rename as SELocAbsoluteS. Simplify Anf. Remove shiftLoc in ClosureConversion.
2021-11-26 12:10:30 +00:00
azure-pipelines[bot]
071bcf7639
update NOTICES file (#11892)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-11-26 11:12:28 +00:00
nicu-da
a1705d669a
participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838)
changelog_begin
changelog_end
2021-11-26 11:12:17 +00:00
Moritz Kiefer
9ff64f7faa
Change daml script’s sleep to sleep for a minimum amount of time (#11886)
* Change daml script’s sleep to sleep for a minimum amount of time

We’ve seen a few flaky test failures where we slept for less than the
expected amount of time which isn’t what we want. We definitely cannot
guarantee an exact sleep time but at least a minimum.

changelog_begin
changelog_end

* Update daml-script/runner/src/main/scala/com/digitalasset/daml/lf/engine/script/ScriptF.scala

Co-authored-by: Remy <remy.haemmerle@daml.com>

Co-authored-by: Remy <remy.haemmerle@daml.com>
2021-11-26 11:12:08 +00:00
Moritz Kiefer
132c277bda
Add a Canton sandbox to the SDK (#11881)
This is only the first step so we have something to use in tests,
eventually `daml sandbox` should map to this Sandbox.

part of #11831

changelog_begin
changelog_end
2021-11-26 11:27:41 +01:00
Moritz Kiefer
68a23436e8
Only run self-service compat job on PRs (#11893)
This is currently failing for the notice file update PR. While we
could try to hack around the git commands to do something sensible, I
don’t really see the point. We really only need this for PRs.

changelog_begin
changelog_end
2021-11-26 10:26:21 +01:00
pbatko-da
c27406cdcf
[DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879
Other changes:
- Fix ErrorCodeDocumentationGeneratorSpec not being in any bazel target.
- (partially) Fix inconsistent usage of deprecated, Deprecation and DeprecatedDocs annotation.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 10:22:14 +01:00
nicu-da
13797229cf
Adapt the compatibility exclusions (#11872)
changelog_begin
changelog_end
2021-11-26 10:07:02 +01:00
Remy
d66ecc90c9
LF: Drop Archive Snapshot for LF < 1.14 (#11820)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 09:18:52 +01:00
tudor-da
abc141b57e
Increase pruning tests timeout (#11891)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-26 08:16:35 +00:00
Andreas Herrmann
66b407467f
Update protobuf docs plugin (#11880)
* Try to upgrade protobuf docs plugin

changelog_begin
changelog_end

* Fix extension number 65020 is already registered

Building `//ledger-api/grpc-definitions:ledger-api-docs` [failed
with](https://github.com/digital-asset/daml/issues/11761#issuecomment-978947565)
```
panic: proto: extension number 65020 is already registered on message google.protobuf.FieldOptions
```

Go dependencies are now pulled in via Gazelle. By default Gazelle will
generate new proto rules for any `.proto` files encountered in third
party Go dependencies. However, many of these already have pregenerated
`.pb.go` files generate with the appropriate configuration.

The problem can be avoided by configuring Gazelle to not generate new
proto rules, but instead use pre-existing `.pb.go` files.

For reference the field number is set in
[go-proto-validators](32a686adf8/validator.proto (L19))
which is an indirect dependency through protoc-gen-doc.

In this case we need to update protoc-gen-validate to v0.6.2 to include
4f41f10dde
which fixes unknown label errors.

* ./fmt

* Expose gRPC status.proto for Haskell bindings

* Update Gazelle to support embedsrcs on Windows

`protoc-gen-doc` relies on `go:embed` file embedding
2dde01902b/resources.go (L8).
Gazelle supports `embedsrcs`, however, it did not generate the attribute
correctly on Windows due to the different directory separator. This is
fixed in https://github.com/bazelbuild/bazel-gazelle/pull/1101.

* Add gazelle to compatibility workspace

It's loaded into `@daml`'s top-level `BUILD` file and ends up being a
dependency of the compatibility workspace as well.

* shift go_googleapis import

* Delete dead code

protobuf is imported transitively.

* Document how to add Go dependencies

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-25 20:51:24 +00:00
Remy
b0dda53868
LF check stable proto with buf and md5sum. (#11888)
The resulting tests should be faster and more reliable

fixes #11443

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 20:23:46 +00:00
mziolekda
056fc52706
Log while processing base64 encoded server key [DPP-761] (#11835)
* Log while processing base64 encoded server key

CHANGELOG_BEGIN
CHANGELOG_END

* address review comments

* demote logs from error to debug

* format
2021-11-25 18:02:35 +00:00
Gary Verhaegen
dbda67ba81
bump JVM in Docker image (#11883)
Please don't depend on this, it breaks all the time.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 18:08:38 +01:00
Kamil Bozek
f69bd688cc
ledger-api-bench-tool: Fix flaky MetricsCollectorSpec (#11750)
* Renamed MetricReporter to MetricFormatter

* Make MetricsCollector communicate via messages only, not by logging

* Logging reports from MetricManger

* Cancel periodic report request when stream is done

* Removed debug sleep

* Fix error when defining reporting period less than one second

* Fixed flaky MetricsCollectorSpec test

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 17:39:30 +01:00
Moritz Kiefer
cb758e8d91
Fix call to experimental interface signatory builtin (#11882)
Turns out if you mistype the name it doesn’t work, who would have
thought.

changelog_begin
changelog_end
2021-11-25 16:33:18 +00:00
Sofia Faro
024400b9e4
Error when fetching the wrong template id (via fetch by interface). (#11862)
* Prevent wrongly typed fetch by interface.

When doing a "fetch by interface" command with a known template id,
error out with a WronglyTypedContract if the fetched contract has
a different template id. This doesn't affect daml, only affects
replays, so it's rather minor. I also enabled the engine test that
caught this.

Part of #11703, follow up to #11836.

changelog_begin
changelog_end

* strengthen test output checks
2021-11-25 16:19:36 +00:00
Sofia Faro
0852c8f6fa
Make DA.List.Total return Optional instead (#11878)
* Make DA.List.Total return Optional instead.

Instead of using ActionFail or CanAbort, return an Optional. The library
user can then convert this into the appropriate error if they wish.

This PR also removes DA.Optional.Total (and the deprecated DA.Maybe.Total)
for having questionable utility.

changelog_begin

- [Daml Standard Library] DA.List.Total functions now return Optional
  instead of being polymorphic in the return type. DA.Optional.Total
  has been removed.

changelog_end

* update Examples.daml
2021-11-25 14:33:48 +00:00
Victor Peter Rouven Müller
df373466dc
[JSON-API] Add query store metrics (#11809)
* Add query store metrics

CHANGELOG_BEGIN

- [JSON-API] added metrics to separately track:
    - time taken to update query-store ACS (from ledger)
    - lookup times for the query store

CHANGELOG_END

* Apply review comment
2021-11-25 13:04:28 +00:00
Sofia Faro
2f8f69e7cb
Drop DA.Next.Set and DA.Next.Map (#11864)
* Drop DA.Next.Set and DA.Next.Map

Fixes #11527. Also removed a bunch of unnecessary CPP, though there's
still a lot more CPP to remove.

changelog_begin

- [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed
  after being deprecated since Daml-LF 1.11

changelog_end

* Drop a deriving MapKey

* update unstable-types test
2021-11-25 13:02:08 +00:00
pbatko-da
5f3a4d2067
[Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867)
Also make section header markers consistent docs' reamde.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 13:46:12 +01:00
tudor-da
cf3ac011ca
[Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828)
* [Self-service error codes] Do not return error code id in metadata

CHANGELOG_BEGIN
CHANGELOG_END

* Do not propagate definite_answer
* Additionally, remove redundant context info from error codes

* Fix tests

* Create IndexDbException as a specialization
* Used to globally define error codes returned by the persistence layer
that are logging
2021-11-25 12:42:29 +01:00
Simon Meier
026b92a8b1
Add gRPC definitions for participant user management service (#11818)
* Introduce definitions for experimental user management service.

Only gRPC definitions are added. No new service is exposed by the Ledger
API server.

CHANGELOG_BEGIN

- [Ledger API] Introduce gRPC definitions for experimental user
  managament service to manage users and their rights for interacting
  with the Ledger API served by a participant node.

CHANGELOG_END
2021-11-25 12:26:47 +01:00
Andreas Herrmann
2fde30d916
Disable writing volatile bits in Scala statsfile (#11875)
Make Scala targets deterministic by disabling writing milliseconds
timestamps into the statsfile.

Before (cache disabled)
```
$ n=1; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ n=2; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ diff -u execlog1.json execlog2.json
--- execlog1.json       2021-11-25 09:53:45.638891575 +0100
+++ execlog2.json       2021-11-25 10:01:52.831225558 +0100
@@ -311392,7 +311392,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile",
     "digest": {
-      "hash": "1c535baa94812c16d33b6b3dcf2dec8e12b77e3a28cd1280ca2c9678840c4de6",
+      "hash": "a7f6f1b168649c1c5346ff92c69f0095b68781690e4b3f33321d8f35af8b3d77",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
@@ -311705,7 +311705,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile",
     "digest": {
-      "hash": "b3af662512a906ec56abfb222f43af189b88b00ae0449bd44c25b7e90689c4db",
+      "hash": "d6c6e2a36754efea4863593a838fbfb500a66e0150d3c3e2e2490ad65c22a2cd",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
@@ -312025,7 +312025,7 @@
   }, {
     "path": "bazel-out/k8-opt/bin/libs-scala/adjustable-clock/adjustable-clock.statsfile",
     "digest": {
-      "hash": "4326088227e5ecf2c21e76ddb6d77c9692ef1689dd146f65ec1e75c235382251",
+      "hash": "8a6b23f3c64ce3ff171772808ca0064af52bc145a32d1b6398de124aeb813c97",
       "sizeBytes": "16",
       "hashFunctionName": "SHA-256"
     }
$ diff -r output1 output2
diff -ur output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile
--- output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile      2021-11-25 09:53:41.706824833 +0100
+++ output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/dependency_analyzer.statsfile      2021-11-25 10:01:49.047160467 +0100
@@ -1 +1 @@
-build_time=2507
+build_time=2522
diff -ur output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile
--- output1/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile    2021-11-25 09:53:39.110780774 +0100
+++ output2/bazel-bin/external/io_bazel_rules_scala/third_party/dependency_analyzer/src/main/scala_version.statsfile    2021-11-25 10:01:46.427115402 +0100
@@ -1 +1 @@
-build_time=4162
+build_time=3946
diff -ur output1/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile output2/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile
--- output1/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile    2021-11-25 09:53:45.282885532 +0100
+++ output2/bazel-bin/libs-scala/adjustable-clock/adjustable-clock.statsfile    2021-11-25 10:01:52.511220053 +0100
@@ -1 +1 @@
-build_time=3158
+build_time=3100
```

After (cache disabled)
```
$ n=3; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ n=4; bazel clean; bazel build //libs-scala/adjustable-clock --execution_log_json_file=execlog$n.json; rsync -aL bazel-bin output$n
$ diff -u execlog3.json execlog4.json
$ diff -ur output3 output4
```

See https://github.com/bazelbuild/rules_scala/pull/1298

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-25 11:01:17 +00:00
Andreas Herrmann
4ed9deda39
Remove xxd from dev-env (#11876)
The corresponding Nix attribute is missing
```
$ xxd
[dev-env] Building tools.xxd...
error: attribute 'xxd' in selection path 'tools.xxd' not found
[dev-env] Build of tools.xxd has failed!
```

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-11-25 11:01:04 +00:00
Gary Verhaegen
eaded41aca
remove mergify (#11866)
It's been working pretty well for us, but our usage is now completely
covered by GitHub's auto-merge feature, which has the advantage of not
requiring us to duplicate our branch protection rules.

This will also reduce our security audit surface as that's one fewer
provider to scan (not that we've found anything wrong with them so far,
but still, less work in the future).

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 11:36:24 +01:00
Andreas Lochbihler
3cd5028b61
fix a few more things in the daml-lf spec (#11851)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2021-11-25 10:46:19 +01:00
Marton Nagy
beca0ee3e9
Refactor StandaloneApiServer factory (#11842)
To enable direct IndexServer building (and enable two step ApiServer build process)
This will be needed for KV WriteService proxy: a wrapper is being build to serve deduplication time conversion, which needs access to completion service.

changelog_begin
changelog_end
2021-11-25 10:12:23 +01:00
Hubert Slojewski
6356f13cde
Properly upgrade gRPC to 1.41.0 (#11858)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 08:50:33 +00:00
tudor-da
f6accd3a2f
Release 1.18 RC2 (#11869)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-25 08:36:05 +01:00
Gary Verhaegen
d858873c44
fix main (#11868)
sorry!

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 23:11:59 +00:00
azure-pipelines[bot]
da8dd7ee2f
rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845)
@nickchapman-da is taking care of 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11844), so they get pushed back to the end of the line.

Please do not merge this before #11844.

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2021-11-24 19:35:46 +00:00
pbatko-da
066da4f30c
[Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 19:23:06 +01:00
Stefano Baghino
258fb65d1c
Document how to deal with HTTP JSON API schema changes (#11336)
* Document how to deal with HTTP JSON API schema changes

changelog_begin
[HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes
changelog_end

Furthermore: uses HTTP JSON API instead of HTTP-JSON API consistently,
drops the capitalization of query store in the documentation and fixes
hyphening in a few places

* Fix title markup
2021-11-24 18:05:30 +00:00
Gary Verhaegen
b8937ad81b
ci: self-service compat test start (#11853)
It should now be possible for anyone to trigger the daily compat build
by adding a `run-full-compat: true` trailer to their commit message.

CHANGELOG_BEGIN
CHANGELOG_END

run-full-compat: true
2021-11-24 18:05:04 +00:00
Gary Verhaegen
de8d15fb1e
fix Nix install on macOS nodes (#11696)
As part of the 2.4 release, the Nix installer has been changed to take
care of the volume setup (which we don't want it to do here). Because
that requires root access, they've decided to make multi-user install
the default, and to disable single-user install.

We could do an in-depth review of the difference and adapt our setup to
use a multi-user setup (we do use the multi-user setup on Linux, so
there's precedent), but as an immediate fix, we can keep our single-user
setup _and_ get the latest Nix by using the 2.3.16 installer and then
upgrading from within Nix itself. This _should_ keep working at least
for a while, as Linux still defaults to single-user.

CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 18:53:13 +01:00
Stefano Baghino
b3d1d40b50
Expose submissionId via the Java bindings (#11839) (#11847)
* Expose submissionId via the Java bindings

Closes #11705

Will need to be forward-ported to `main` once merged.

changelog_begin
[Java Bindings] submissionId is now exposed via the bindings, see issue #11705
changelog_end

* Address https://github.com/digital-asset/daml/pull/11839#discussion_r755445100
2021-11-24 17:38:56 +01:00
Remy
86da6e8eef
LF: Test scala interface type checking (#11833)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 15:58:45 +00:00
Gary Verhaegen
5f52f00afb
increase linux cluster size (#11860)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 15:18:56 +00:00
Sofia Faro
5c12d757f4
Add a guard when exercising by interface. (#11836)
* Add a guard when exercising by interface.

This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.

It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.

I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)

changelog_begin
changelog_end

* scalafmt

* Feedback and fix matches

* Update comments, we are always going to abort the transaction

* Raise WronglyTypedContract in SBGuardTemplateId.

* rebase and fix parser

* restore ANF

* scalafmt
2021-11-24 14:22:40 +00:00
fabiotudone-da
7c3a2a7801
Add a new KV submission failure error (#11854)
CHANGELOG_BEGIN
[Integration Kit] Add a new SUBMISSION_FAILED internal error
CHANGELOG_END
2021-11-24 13:10:36 +00:00
Robert Autenrieth
aebc5a734f
All packages must be valid (#11850)
changelog_begin
changelog_end
2021-11-24 13:10:27 +00:00
nickchapman-da
0374843143
speedy compilation benchmark (#11852)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 13:10:18 +00:00
Remy
393893a601
LF encoder: make package validation optional (#11849)
CHANGELOG_BEGIN
CHANGELOG_END
2021-11-24 13:53:52 +01:00