Commit Graph

5845 Commits

Author SHA1 Message Date
Andreas Herrmann
bd09e8265d
Require authorization on DAR upload endpoint (#8193)
* Test authentication on upload_dar endpoint

changelog_begin
changelog_end

* require authentication on upload_dar endpoint

* push Directive into auth

* Fully upload request before auth redirection

* Make HTTP entity upload parameters configurable

changelog_begin
changelog_end

* Shorten help message

https://github.com/digital-asset/daml/pull/8193#discussion_r538428368

* maxHttpEntityUploadSize as Long

https://github.com/digital-asset/daml/pull/8193#discussion_r538431773

* use DefaultMaxInboundMessageSize for DefaultMaxHttpEntityUploadSize

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 18:09:06 +01:00
Samir Talwar
c588b5cc34
kvutils: Extract a SimplePackage class from TestHelpers. [KVL-759] (#8203)
* kvutils: Simplify KVTest a little.

* kvutils: Split out a SimplePackage manager from TestHelpers.

In an effort to clean up the TestHelpers and associated calls, I have
extracted a class that takes the additional contract data type _once_.
It also provides a useful place to move associated methods and values.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-08 16:08:30 +00:00
Samir Talwar
f7465f0646
kvutils: Extract out a trait from StateUpdates. (#8201)
This allows us to mock the object without resorting to mocking final
classes, which is _hella_ flaky.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-08 15:33:35 +00:00
Moritz Kiefer
f66166ec95
Release 1.8 RC (#8200)
changelog_begin
changelog_end
2020-12-08 16:20:37 +01:00
Remy
8f3c6a4494
LF: Add version directly in GenNode (#8154)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-08 16:12:12 +01:00
Moritz Kiefer
38455e8ca9
Fix Scala version overwrite (#8197)
The old format changed and resulted in that silently being ignored.

changelog_begin
changelog_end
2020-12-08 13:52:41 +00:00
Samir Talwar
fdde69a599
kvutils: Throw if CommitContext#read encounters missing input state. [KVL-757] (#8194)
* kvutils: Throw if CommitContext#read encounters missing input state.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Revert CommitContext#get.

* ledger-on-memory: The pre-execution tests are no longer flaky.

* kvutils: Update the Scaladoc for CommitContext to mention exceptions.

* kvutils: Moar CommitContext testing.
2020-12-08 13:25:11 +00:00
Samir Talwar
73514c53ea
kvutils: Fix a flaky batching queue test. (#8199)
This changes the API so that we can wait for the queue to finish before
asserting that it is, indeed, finished.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-08 13:17:26 +00:00
tudor-da
26b1b3dd13
Disable asynchronous commits in JdbcIndexer (#8195)
CHANGELOG_BEGIN
[JdbcIndexer] Asynchornous commits are disabled.
CHANGELOG_END
2020-12-08 13:05:42 +00:00
Moritz Kiefer
9447fdadca
Clarify DAML Script docs (#8198)
"UI testing" has caused some confusion since DAML Script clearly
doesn’t replace something like puppeteer.

changelog_begin
changelog_end
2020-12-08 12:54:15 +00:00
Moritz Kiefer
880d290285
Publish DAML on SQL Docs (#8191)
changelog_begin
changelog_end
2020-12-08 13:49:37 +01:00
Samir Talwar
0ee2159218
KVUtils: If there is a missing input state during model conformance validation, report "Disputed". [KVL-757] (#8180)
* kvutils: If there is a missing key, print the key.

* kvutils: Log retries during pre-execution.

* kvutils: Handle `Err.MissingInputState` in model validation.

CHANGELOG_BEGIN
CHANGELOG_END

* kvutils: Fetch contracts specified by contract keys for pre-execution.

This ensures that we can generate an appropriate read set even if
there's contention.

* kvutils: Make TestHelper a little more consistent.

* kvutils: Test generating read sets for inconsistent contract keys.

I have verified that this fails on `master` and passes on this branch.

* kvutils: Pull out helpers from PreExecutingSubmissionValidatorSpec.
2020-12-08 11:14:28 +00:00
Bernhard Elsner
7f81db4fc4
Add build rule for DAML Driver for PostgreSQL docs (#8188)
* Add build rule for DAML Driver for PostgreSQL docs

CHANGELOG_BEGIN
CHANGELOG_END

* Reimplement some JS functions

* fmt

changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-12-08 10:19:48 +00:00
Andreas Herrmann
59dae3c97b
Handle admin claims in OAuth2 test server (#8185)
* Middleware test resources

- Make OAuthServer accessible
- Don't access suiteResource directly in the test-suite

* OAuth2 test server test resources

- Make Server accessible
- Don't access suiteResource directly in the test-suite

* Use a deny list in OAuth2 test server

It is simpler to configure a set of disallowed parties and extend it on
demand than switching between blanket approval and allow list modes.

changelog_begin
changelog_end

* Handle admin claims in test server

Requests with admin claims are granted by default and denied if admin
requests have been disallowed.

* reset admin state in test fixtures

* test admin tokens in OAuth2 test server

* test admin claims in OAuth2 middleware

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 10:16:29 +00:00
Andreas Herrmann
e1b1e4d646
Wait for trigger shutdown before restart (#8186)
changelog_begin
changelog_end

This is to avoid a race condition where the old trigger runner may not
yet have completed shutdown by the time that the token has been
refreshed and we attempt to start a new trigger runner.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-08 09:43:51 +00:00
Moritz Kiefer
7e05dc7932
Upgrade rules-scala and scalatest (#8187)
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.

This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.

Apologies for the giant PR, I don’t see a way to keep it smaller.

changelog_begin
changelog_end
2020-12-08 06:59:23 +01:00
Brian Healey
ca294eb14d
add blackduck scan to run on master (#6130) (#8161)
* add blackduck scan to run on master (#6130)

* add blackduck scan

* disable go scanning
exclude entire language-support/ts directory for node scanning
break to multiple lines to make command line params easier to parse

* Increase timeout for blackduck binary scan

* update blackduck scan config

* remove some exclusions, force python3

* exclude GO until path to go executable can be resolved

* added readme explanation of why we want this file

* fail in case of policy violation

* ensure haskell bazel scan completes before running second round scan for bazel jvm and node and other langs

* trigger notices file gen to ensure BOM complete

* remove trailing end of lines

* run with latest detect version and unique code location name changes to wrapper script

* Add blackduck to daily compat job

* DO NOT MERGE: condition false to disable other jobs for testing

* remove parameters not available to cronjob

* Revert changes to regular CI pipeline

CHANGELOG_BEGIN
CHANGELOG_END

Signed-off-by: Brian Healey <brian.healey@digitalasset.com>

* Do not get branch name from variable

* Upgrade com.fasterxml.jackson.core:jackson-databind to 2.12.0 to address security vulnerability

* Remove disabling of other jobs, set to branch to be used on prod runs

* Apply suggestions from code review

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Address code review comments

* Updated NOTICES file

* Run bazel build, update NOTICES file

* Correct dade-assist

* do not have perms to pipe to dev/null

* Add md file explaining how to update NOTICES file

* Add instructions for running blackduck locally

* Add a link to full security-blackduck readme

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2020-12-07 19:59:39 +00:00
Andreas Herrmann
4ac7b3f22d
Separate auth API from implementation and testing code (#8183)
* Rename and separate auth targets and packages

Split out the API types `Request`/`Response` into their own Bazel
targets, so that the trigger service does not have to depend on the full
middleware and OAuth2 test server targets.

Also rename packages: Generic auth code goes into `com.daml.auth`,
OAuth2 specific code goes into `com.daml.auth.oauth2` or
`com.daml.auth.middleware.oauth2`.

changelog_begin
changelog_end

* Remove empty test-case

This is a left-over from #7226.

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-12-07 16:17:46 +01:00
Robert Autenrieth
56f5cf107b
Prepare the ledger API for multi-party submissions [KVL-710] (#8125)
* Add multi-party submissions to the ledger API

CHANGELOG_BEGIN
* [Ledger API] Command submission requests now contain new optional
  fields used for multi-party submissions. Such submissions currently
  return UNIMPLEMENTED errors, they will be enabled in the future.
CHANGELOG_END

* Adapt Haskell bindings

* Handle the new ledger API fields

* Fix SubmitAndWaitDummyCommand

* Remove unused methods

* Redesign multi-party auth tests

* Remove direct access of request party

* Apply review comments

* Improve protobuf comments

* Multi-party tracker map

* Fix validation logic

* Consistent metric naming
2020-12-07 16:17:05 +01:00
Sofia Faro
84e5881236
Add serializability check for exceptions. (#8151)
* Add serializability check for exceptions.

Part of #8020

changelog_begin
changelog_end

* typo
2020-12-07 14:44:24 +00:00
Remy
ab43da3f7a
Speedy: Add a built to optimize cons with 1 elems (#8177)
We have a builtin SBConsMany use to concatenate a list with a prefix
of variable size. Its appear that most of the concatenation happens
with only one element. For this reason we add a new builtin to handle
the most efficiently possible the case with 1 element.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-07 15:13:33 +01:00
Richard Kapolnai
158e866f24
Improve consistency of describing consuming and nonconsuming choices (#8179)
* slightly improve consistency of describing consuming and nonconsuming choices

* CHANGELOG_BEGIN
CHANGELOG_END
2020-12-07 14:33:56 +01:00
Samir Talwar
88cb4c0981
daml-lf: Include position information in the testing parsers. (#8182)
So it's easier to figure out where I've gone wrong.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-07 12:33:00 +00:00
Moritz Kiefer
4e8a7f420e
Deprecate Sandbox persistence (#8181)
* Deprecate Sandbox persistence

changelog_begin
changelog_end

* Update docs/source/tools/sandbox.rst

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* Include in component statuses

changelog_begin
changelog_end

* Fix daml on sql Name param

changelog_begin
changelog_end

* make it compile

changelog_begin
changelog_end

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2020-12-07 12:08:36 +00:00
dependabot[bot]
6f2d4f9aaf
Bump highlight.js from 10.4.0 to 10.4.1 in /language-support/ts/packages (#8176)
Bumps [highlight.js](https://github.com/highlightjs/highlight.js) from 10.4.0 to 10.4.1.
- [Release notes](https://github.com/highlightjs/highlight.js/releases)
- [Changelog](https://github.com/highlightjs/highlight.js/blob/master/CHANGES.md)
- [Commits](https://github.com/highlightjs/highlight.js/compare/10.4.0...10.4.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-12-07 12:31:34 +01:00
Samir Talwar
5943e1ae8d
daml-lf: Simplify types in the TransactionBuilder test helper. (#8178)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-07 09:46:24 +00:00
Richard Kapolnai
78f2fed0ad
Doc fix minor issues in intro (#8174)
* fix referring to a transaction
CHANGELOG_BEGIN
CHANGELOG_END

* fix typo

* remove ambiguous statement

* rename setupResults to setupResult

* typo

* Reformat the two principles.

* typo

* revert removal
2020-12-07 09:44:10 +00:00
Richard Kapolnai
49d1f1c37f
typo: doubling words (#8173)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-07 09:53:21 +01:00
Richard Kapolnai
9d2dd51684
Doc: fix minor issues in intro 2 (#8175)
* remove inadequate picture

* simplify example

* fix format in comment

* typo

* add signature screenshot

* dummy whitespace
CHANGELOG_BEGIN
CHANGELOG_END

* revert dummy whitespace
2020-12-05 16:23:12 +01:00
Robert Autenrieth
35d52e956e
Prepare index database for multi-party submissions [KVL-700] (#8035)
* Support multiple submitters in LedgerEntry

CHANGELOG_BEGIN
CHANGELOG_END

* Adapt the database schema

* Fix party formatting

* Add some tests

* Change logic for when requesters see commands

* Disable test that can't work

* Handle empty sets in array intersection

* Don't query completions as an empty set

... this should never happen in practice

* Add test for filtering command IDs

* Enable multiple parties in command deduplication (#8092)

* Enable multiple parties in command deduplication

CHANGELOG_BEGIN
CHANGELOG_END

* Update ledger/participant-integration-api/src/main/scala/platform/store/dao/LedgerDao.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update ledger/participant-integration-api/src/main/scala/platform/store/dao/LedgerDao.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Apply review suggestions

* Remove useless function

* Fix EventsTablePostgresql

* Apply review comments

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-12-04 21:47:52 +01:00
fabiotudone-da
29dd5909b8
Document participant pruning for operators [KVL-421] (#8138)
* Document participant pruning for operators

CHANGELOG_BEGIN
CHANGELOG_END

* Add references

* Add references and authorization info

* Address review comments

* Keep paragraphs on the same line

* Fix link to Pruning Service

* Add services section for Pruning Service

* Add Pruning Service to service diagrams

* Fix leftover

* How to determine a suitable pruning offset

* Expand determining a suitable offset

* Expand API impacts

* Expand limitations

* Address review comments

* Address review comments
2020-12-04 19:35:31 +01:00
fabiotudone-da
943f8b53ad
Remove "contract visible to submitter" check in KV utils [KVL-725] (#8169)
* removeUnnecessaryNodes -> trimUnnecessaryNodes

* Remove contract visibility check in kvutils

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-04 17:36:28 +01:00
Moritz Kiefer
8d6f1c929b
Migrate Navigator from TSLint to ESLint (#8172)
* Switch config from tslint to eslint

changelog_begin
changelog_end

* Rename disable comments

changelog_begin
changelog_end

* Fix all the lints

changelog_begin
changelog_end
2020-12-04 17:02:13 +01:00
Bernhard Elsner
ec41bb2652
Small sizing and alignment fixes (#8171)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-04 15:16:50 +00:00
Bernhard Elsner
0059844349
Fix Next button padding (#8168)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-04 13:38:31 +00:00
Sofia Faro
c3d99c1695
Dont look for daml.yaml during SDK install. (#8167)
* Dont look for daml.yaml in SDK install.

Unless you're running "daml install project".

Fixes #5720

changelog_begin
changelog_end

* use pattern synonym for raw install target
2020-12-04 13:07:30 +00:00
Bernhard Elsner
503600f5d0
Docs style alignment (#8164)
* Update landing page icons

CHANGELOG_BEGIN
CHANGELOG_END

* Fix menu arrow alignment

* Fix the colour of the icons

* Improve landing page specing
2020-12-04 13:43:44 +01:00
Gary Verhaegen
46b7decbca
notify_user: consider changelog entries (#8166)
The current implementation of the notify_user job sometimes reports
success while the build has actually failed. Azure does not provide a
way to query the overall state of the current build, so a general
solution to this problem does not seem possible (see #6796 for an
example of a failed attempt). However, all reported cases were
specifically about the `check_changelog_entry` job, which we can easily
query for, so this PR does that.

Note: originally pushed without a changelog entry to test new
notification mechanism.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-04 12:16:34 +00:00
azure-pipelines[bot]
c86b7a561f
update compat versions for 1.8.0-snapshot.20201201.5776.0.4b91f2a6 (#8132)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2020-12-04 13:01:34 +01:00
Richard Kapolnai
6a9a32fb97
Rename scenarios to scripts (#8147)
* Rename scenarios to scripts
CHANGELOG_BEGIN
CHANGELOG_END

* Refer to test scripts

* avoid referring to RT and LT in tests

* remove obsolete restrictions

* fix underline

* revert accidental type annotation

* fix typo

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update docs/source/daml/intro/5_Restrictions.rst

Co-authored-by: Richard Kapolnai <richard.kapolnai@digitalasset.com>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2020-12-04 10:02:10 +00:00
Bernhard Elsner
f0b98a85b9
Docs style alignment (#8134)
* Flashier top-level menu

* Flashy buttons

CHANGELOG_BEGIN
CHANGELOG_END

* Increase top-level nav font-size

* Make the entire landing page box clickable

* Fix code block alignment and copy button with line numbers

* Fix code blocks with line numbers on mobile

* Reengineer code blocks properly

* More fixes to code-blocks

* Add a vertical line after line numbers
2020-12-04 08:50:08 +01:00
Moritz Kiefer
e0e317e05d
Avoid favicons-webpack-plugin in Navigator (#8159)
We really don’t need 5 different favicons here so just inline the same
one we use in create-daml-app and on our docs (32x32). This allows us
to get rid of tons of dependencies including a native dep on sharp.

changelog_begin
changelog_end
2020-12-03 18:15:56 +00:00
nickchapman-da
3b575c7d89
Update instructions for working with ghc-lib (#8158)
* Update instructions for working with ghc-lib: Describe how to use a windows ad-hoc machine to run the `bazel` pin step.

changelog_begin
changelog_end

* address comment
2020-12-03 17:52:16 +00:00
Gary Verhaegen
0e0b9ab8a4
JSON API: document encoding flags (#8157)
CHANGELOG_BEGIN
CHANGELOG_END
2020-12-03 17:40:32 +00:00
Robin Krom
eb8114728a
js codegen: better error and peer-dependencies (#8131)
We add the daml-types/daml-ledger libraries as peer-dependencies instead
of dependencies. This will assure that the library is provided by the
importing library and avoid the situation when two different versions of
these libraries are imported by the generated and the user code.

This fixes #8101.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-03 18:21:11 +01:00
Moritz Kiefer
4cfb360da2
Bump remaining Navigator frontend deps (#8156)
This finally gets up to the point where almost all deps are on the
latest version (according to `yarn outdated`). The one exception is
webpack since at least some of our webpack loaders don’t work with
webpack 5 yet.

Another issue is that tslint is deprecated and should be replaced by
eslint. I’ll tackle that as a separate PR.

changelog_begin
changelog_end
2020-12-03 17:15:21 +00:00
Samir Talwar
38de5b1eb3
daml-lf: Remove aliases that have been deprecated since v1.4. (#8155)
CHANGELOG_BEGIN
- [DAML-LF] Some types that were deprecated in DAML 1.4 have been
  removed. Please fix any deprecation warnings by using the new types
  before upgrading.
CHANGELOG_END
2020-12-03 17:07:42 +00:00
Samir Talwar
17c8ce788f
ledger-on-memory: Mark pre-execution conformance tests as flaky. (#8153)
This is a temporary measure while we figure out what's wrong.

CHANGELOG_BEGIN
CHANGELOG_END
2020-12-03 15:47:49 +00:00
Richard Kapolnai
f899a3e19c
Fixing minor documentation errors (#8144)
* fix minor typos
CHANGELOG_BEGIN
CHANGELOG_END

* Remove notes and comments on pragma

* minor fix: remove "again" again

* fix various typos

* remove unexplained "bank" references

* replace LET with LT

* fix typo

* remove "again"

* fix grammar

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>

* adjust line numbers to include

Co-authored-by: Bernhard Elsner <40762178+bame-da@users.noreply.github.com>
2020-12-03 15:28:34 +00:00
nickchapman-da
0867605e1a
Don't loose signatures in template let binding (#8136)
* update ghc-lib patch

* update stack-snapshot.yaml

changelog_begin
changelog_end

* bazel run @stackage-unpinned//:pin (on linux) and update stackage_snapshot.json

* testcase to check type signatures in template-lets are not ignored

* update to ghc-lib commit hash following merge

* update snapshot

* update snapshot after pin on windows

* update stack pin on linux

Co-authored-by: Nick Chapman <nchapman@digitalasset.com>
2020-12-03 15:08:02 +00:00