Commit Graph

8724 Commits

Author SHA1 Message Date
Stefano Baghino
52110e31a3
Remove Extractor (#12188)
* Remove Extractor

Extractor is being removed after a long time being in Labs status.

This should improve the flakiness on CI.

🔥

changelog_begin
Extractor has been removed from the SDK
changelog_end

* Remove Extractor documentation

* Remove Extractor from CODEOWNERS

* Remove Extractor references in Daml-LF build files

* Remove Extractor references in the Daml SDK assistant

* Remove Extractor from the SDK

* Remove Extractor reference from CONTRIBUTING.md
2021-12-17 13:43:23 +00:00
Hubert Slojewski
c4b5de609a
kvutils: Remove references to Daml-LF Archive from kvutils protos [KVL-1167] (#12168)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-17 13:24:46 +01:00
Moritz Kiefer
1329fde631
Release another 2.0 snapshot (#12182)
This time with the new maven host.

changelog_begin
changelog_end
2021-12-17 11:42:25 +00:00
Moritz Kiefer
5f688dd704
Cache localContract calls in contractKeyInputs (#12183)
changelog_begin
changelog_end
2021-12-17 11:37:24 +00:00
Moritz Kiefer
adb1f0e716
Split up docs postprocessing & release in split-release (#12176)
* Split up docs postprocessing & release in split-release

Follow up to #12172

The buld in the separate assembly should now only have to run the
sphinx-build step and then merge it with the non sphinx sources and
run lualatex on the pdf sources with the pdf fonts which is hopefully
simple enough that we can easily replicate it.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-12-17 11:22:16 +01:00
nicu-da
2ba3d60947
Add conversions for CompletionStage to Future which unwraps failures (#12165)
changelog_begin
changelog_end
2021-12-17 09:34:20 +00:00
Moritz Kiefer
d230cc88f6
Switch to new sonatype host (#12179)
They migrated our account so hopefully this should work without any
other changes and fix our publishing issues.

I’m keeping the long timeout for now since I don’t know what an
appropriate timeout is.

changelog_begin
changelog_end
2021-12-17 09:19:02 +01:00
tudor-da
ea7d49d4d9
Implement Sandbox-on-X runner [DPP-823] (#12150)
* Implement Sandbox-on-X runner

CHANGELOG_BEGIN
CHANGELOG_END

* Address Marton's comments

* Extracted AkkaSubmissionsBridge

* Addressed second round of review comments
2021-12-17 07:38:16 +00:00
Sergey Kisel
4c8be783b2
use domain.Offset instead of String in ContractsFetch functions (#12160)
* Initial commit

CHANGELOG_BEGIN
CHANGELOG_END

* formatting fix

* fix up graph DSL whitespace

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-12-16 18:14:39 +00:00
Stefano Baghino
42adfdc857
Get rid of deprecated components (#12167)
* Get rid of deprecated components

Removes:
- mentions of the Node.js bindings
- mentions of the Scala bindings
- usage of the Scala codegen as an SDK tool
- Java bindings "reactive components"

changelog_begin
The Node.js bindings are no longer supported as part of the Daml SDK
The Scala bindings are no longer supported as part of the Daml SDK
The Java bindings reactive components are no longer supported as part of the Daml SDK
The `daml codegen` command has dropped support for Scala
changelog_end

* Fix typo noticed by @cocreature

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Address https://github.com/digital-asset/daml/pull/12167#issuecomment-995714093

* Remove quickstart-scala from the SDK tarballs

* Fix failing codegen test

* Address https://github.com/digital-asset/daml/pull/12167#pullrequestreview-833991243

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-12-16 16:55:19 +00:00
Remy
7ecc12645e
LF: make Struct lookup logarithmic (#12170)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 16:48:45 +00:00
akshayshirahatti-da
4745768ad4
[Oauth2-Middleware] Changes to introduce HOCON+pureconfig for oauth2-middleware (#12061)
* Changes to introduce HOCON+pureconfig for oauth2-middleware

CHANGELOG_BEGIN
CHANGELOG_END

* remove Cli arg parsing code + cleanup based on code review

* addition of a minimal config and changes to README.md

* keep existing cli args, but load from config file if provided

* fix broken docs build

* make tests OS independent

* Fail/error on supplying both config file and cli opts for startup, address code review comments
2021-12-16 16:36:32 +00:00
Sofia Faro
e52469c1dc
Add a test that mkMethod call causes a type error. (#12175)
Relevant discussion: https://github.com/digital-asset/daml/pull/12171#discussion_r770553009

changelog_begin
changelog_end
2021-12-16 15:52:44 +00:00
Moritz Kiefer
97e3a1a557
Refactor docs build (#12172)
* Refactor docs build

This is in preparation for the split docs build where we publish our
sphinx sources here and then build the final docs combined with the
Canton docs in the assembly repo.

Apologies for the messy diff. The idea here is the following:

1. We bundle together all files used as sphinx inputs in
sphinx-source-tree. This will be what we publish.
2. We factor out the calls to sphinx-build for the pdf and html
build to be as generic as possible and share the code.
3. The existing rules for the html build & pdf docs don’t invoke
sphinx anymore and instead only keep the postprocessing (copying other
html files or calling lualatex).

In a followup PR I’ll also simplify the postprocessing steps and then
finally publish those artifacts as part of our build.

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-12-16 15:37:21 +00:00
Sofia Faro
8b05a533ce
Refactor daml start options, add true/false to yes/no/auto flags. (#12169)
* Refactor daml start & add true/false to yesNoAuto.

- Refactor the way we pass arguments to daml start. We were relying on
  positional arguments with newtypes, but this is super cumbersome. I
  changed it to a RecordWildCards-style approach, where we don't need
  quite so many newtypes, and no more positional arguments.

- "--start-navigator" flag had some custom logic to accept "true" and
  "false". I don't see why we can't just accept "true" and "false"
  anywhere we use the "yes/no/auto" flags, so I just changed that and
  got rid of the custom logic.

- The way "auto" was handled for this flag was incorrect, since "auto"
  is supposed to be equivalent the default, i.e. not passing any
  flag. I changed it so auto is equivalent to not passing an argument.
  (I.e. it looks in daml.yaml for the start-navigator option).

changelog_begin
changelog_end

* dont pass in shutdownStdinClose to runStart
2021-12-16 14:40:36 +00:00
Robin Krom
1ed02369eb
interfaces: fix mkMethod erasure (#12171)
We erase the call to `mkMethod` in `convertExpr` instead of pattern
matching against the binding. This fixes a bug when the method body
contained statements using typeclass dictionaries such as
`do [2] === [2]`.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 15:22:46 +01:00
fabiotudone-da
2940a7b5c2
[ledger-api-test-tool] - existing_submission_id and completion_offset assertions for ALREADY_EXISTS rejections in command dedup ledger API tests [KVL-1020] (#12140)
changelog_begin
changelog_end
2021-12-16 06:11:39 -08:00
Moritz Kiefer
98d4e177cf
Release 2.0 snapshot (#12164)
This time with the increased Maven timeout which is hopefully enough
to get it to succeed.

changelog_begin
changelog_end
2021-12-16 12:21:48 +00:00
Remy
39ee326a51
LF: Drop wrong toArraySeq method from ImmArray. (#12163)
The method is not correctly implement, as it should slice the
underlying array before returning it. Since it is not possible to
implement it in constant time, we simply drop it, in favor in other
conversion.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 10:18:52 +00:00
Moritz Kiefer
101356ba36
Bump sonatype timeout to absurdity (#12162)
changelog_begin
changelog_end
2021-12-16 09:38:03 +00:00
Simon Meier
64e228b72b
integration-kit: remove all its docs (#12157)
CHANGELOG_BEGIN
- [Daml Integration Kit]: removed from Daml 2.0
CHANGELOG_END
2021-12-16 10:30:19 +01:00
azure-pipelines[bot]
1189dd5915
update NOTICES file (#12161)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-16 08:54:29 +00:00
Marton Nagy
115fbba799
Introducing DbSupport, pulling up DbDispatcher building in the factory hierarchies (#12152)
This PR prepares the landscape for seamless integration of Index DB with non IndexService related
front components like UserManagementStore.
DbSupport is a simply a DbDispatcher and a StorageBackendFactory. Creation of it comes also in Flyway migrated flavor for capturing common need.
This is a pure refactoring PR: no behavior/feature changes

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 08:49:20 +01:00
Samir Talwar
9d00861b4e
resources-akka: Wait for the bounded source queue to finish. (#12142)
* resources-akka: Wait for the bounded source queue to finish.

Otherwise, we may get submissions after dependencies have shut down.

CHANGELOG_BEGIN
CHANGELOG_END

* resources-akka: Simplify the interface and use clearer type param names.
2021-12-15 20:42:06 +00:00
Brian Healey
bb89f369c0
revert to use synopsys detect 7.1.0 (#12155)
* revert to use synopsys detect 7.1.0 run-full-compat: true

CHANGELOG_BEGIN
CHANGELOG_END

* run on branch run full compat

run-full-compat: true

* run on branch run full compat

run-full-compat: true

* revert changes to run branch
2021-12-15 14:39:28 -05:00
Simon Meier
51df74490b
user management: complete authentication tests (#12143)
* user management: complete authentication tests (#12062)

Also incorporate naming suggestion wrt 'createUserByAdmin' from @mziolekda

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-15 14:40:05 +00:00
Robin Krom
5af58a1652
interfaces: doc generation for interface methods (#12096)
This adds documentation generation for interface choices and methods.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-15 13:57:09 +00:00
Moritz Kiefer
506eee16f5
Extend VSCode extension troubleshooting section (#12154)
I’ve had to look this up for the second time today so documenting it
seems very sensible.

changelog_begin
changelog_end
2021-12-15 13:56:30 +00:00
Moisés Ackerman
149474f3cb
Drop scenarios from the user-facing SDK (#12128)
* Add --enable-scenarios hidden flag
* Fail LFConversion when a scenario is defined and --enable-scenarios=no
* Remove unused optIsGenerated=True test cases
* Refactor getIntegrationTests
* Add no-scenarios-enabled tests
* Add enable-scenarios flag to scenario-service server
* Add test ensuring that scenario-returning functions are still allowed
* Remove 'enable scripts' option (--daml-script)
* Add --scripts synonym to --scenarios
* Rephrase scenarios flags descriptions

changelog_begin
changelog_end
2021-12-15 13:39:14 +01:00
azure-pipelines[bot]
9a1fa5ab10
release 2.0.0-snapshot.20211214.8692.0.63940872 (#12147)
* release 2.0.0-snapshot.20211214.8692.0.63940872

This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@cocreature is in charge of this release.

Commit log:
```
6394087280 Add transaction statistics to CompletionInfo (#12139)
a03df88f7c DPP-721 Remove old ACS implementation (#12045)
81990bc8b3 Fix #12133: correct error for tokens with invalid user-ids (#12134)
717772850d Update Logback to 1.2.8 (#12141)
8c1173c8f1 User management application id defaulting (#12100)
259be4340b kvutils: Move `TransactionUtils` from `pkvutils` [KVL-1166] (#12138)
83bb84a73f kvutils: Change `NodeId` from `String` to a value class (#11943)
798137964b Add requires keyword for syntax highlighting (#12113)
d1fb5e442c interfaces: Implement 'requires' syntax (#12105)
01364f2863 Enable Daml Script user management tests (#12130)
75863c392f Fix #12129: allow users' access to their own info (#12132)
f146bc814b Escape daml-lf tracelog messages (#12116)
6c7e33cafe Add Ledger API feature descriptor for user management. (#12095)
9a931854b2 add docs for command dedup (#11928)
f07596cb38 KV: `VALIDATION_ERROR` is not an internal error as it can also be caused by pruned contracts [KVL-1207] (#12108)
f223528bfb Implement PoC of user management for Ledger API server (fix #12014) (#12063)
787dccb3d5 Bump TriggerServiceTest timeout (#12127)
78a0677fb1 Fix command in triggers docs (#12123)
77a37cf213 Drop damlc code for LF <= 1.11 (#12121)
caa8072d91 Drop unused primitives for interface contract id conversion (#12114)
82465e2e85 Drop Decimal completely from damlc (#12120)
e1a2da0771 [ledger-api] - Add deduplication_offset as a valid deduplication period for commands [KVL-1172] (#12041)
9ff8851488 Fix a docs display issue with Safari and Firefox (#12035)
36106c39de Drop featureNumeric (#12119)
4cc88a2fc3 `ledger-api-bench-tool` - item rate SLO [DPP-790] (#12033)
7f76cf1bdf Drop featureAnyType and featureTypeRep (#12112)
c5708c96c5 es: mitigate log4j vuln (#12118)
b6b6ecd669 repair ES cluster (#12117)
937849b101 Define asciiToLower and asciiToUpper in DA.Text (#12109)
124f8854f0 daml-docs: Refactor context type (#12111)
778def6f48 Drop non-type synonym code from LF conversion (#12110)
25ae69eb01 [DPP-657] Make error codes docs nicer (#12079)
8c87a0e327 LF: Factorize interpretation loop emulation spread in multiple tests. (#12102)
9f8e640b1a Use ExecutionContext.parasitic instead of DirectExecutionContext. (#11783)
9a01bb9b86 Fix filters in ExceptionRaceConditionIT (#12106)
3fae3b2e3e Stop flyway from printing sensitive jdbc string at info level (#12088)
ae8c2788b8 Split release test with cross-platform damlc (#12107)
8011d602c8 Support multiple participant in daml script’s user management (#12091)
1989a2def9 LF: Check non circularity of `requires` Interface field
```
Changelog:
```
- [Ledger API]: as a convenience when using JWT authentication, a
  request's application_id field can be left empty to ask the server to
  set the field to the id of the authenticated user or the application-id
  in the custom token. There are three services that currently specify
  application_id's in the request: CommandService,
  CommandSubmissionService, and CommandCompletionService.
- [Ledger API]: introduce a feature descriptor for whether the ledger supports user management
Documentation on the new command deduplication mechanism
- [Integration Kit] "VALIDATION_ERROR" is not an internal error anymore as it can also be caused by pruned contracts
- [User Management]: add support for managing participant node users and authenticating
  requests as these users using standard JWT tokens.
[ledger-api] - Add `deduplication_offset` as a valid type for the command deduplication period
- [Integration Kit] - A minimum count rate objective can be defined for a stream in the ledger-api-bench-tool
```

CHANGELOG_BEGIN
CHANGELOG_END

* Update LATEST

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-12-15 12:06:00 +01:00
nicu-da
bb11399091
kvutils - use the command readAs parties alongside the actAs parties (#12136)
changelog_begin
[participant-state] - SubmitterInfo now includes the readAs parties
changelog_end
2021-12-15 02:29:32 -08:00
Moritz Kiefer
ffe05aab3d
Apply grpc-max-message-size in both directions in the script service (#12149)
Previously, it was only applied for messages towards the script
service but not for messages towards the Haskell client. Now we
increase the max size in both directions.

See
https://discuss.daml.com/t/increase-max-message-size-in-daml-studio/3573/9
for the forum thread that raised this.

changelog_begin

- [Daml Studio/Daml Test] The `grpc-max-message-size` field in
  `daml.yaml` now also applies to the message that contains the result
  of a script.

changelog_end
2021-12-15 09:33:55 +00:00
Simon Maxen
6394087280
Add transaction statistics to CompletionInfo (#12139)
* Add transaction statistics to CompletionInfo

changelog_begin
changelog_end

* Update with review comments

* Reformat

* Remove issue link
2021-12-14 22:50:01 +00:00
Robert Autenrieth
a03df88f7c
DPP-721 Remove old ACS implementation (#12045)
* Remove old ACS implementation

changelog_begin
changelog_end

* Remove QueryParts.ByLimit
2021-12-14 22:54:44 +01:00
Simon Meier
81990bc8b3
Fix #12133: correct error for tokens with invalid user-ids (#12134)
* Fix #12133: correct error for tokens with invalid user-ids

- also changes the error for a token referring to an unknown user from `UNAUTHENTICATED` to `PERMISSION_DENIED`

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-14 21:01:12 +00:00
Gerolf Seitz
717772850d
Update Logback to 1.2.8 (#12141)
This is to remove a JNDI and JDBC related vulnerability.
See https://jira.qos.ch/browse/LOGBACK-1591.

The vulnerability is only exploitable if the logback configuration
already makes use of JNDI lookups. This is _NOT_ the case for
logback configuration files shipped with any Daml components.

Nevertheless, it is good practice to upgrade dependencies known to be
vulnerable.

Regenerating the bazel dependency files also revealed that a previous
update to gRPC 1.42.0 was committed without running
`bazel run @unpinned_maven//:pin`.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-14 18:37:16 +00:00
Simon Meier
8c1173c8f1
User management application id defaulting (#12100)
user management: add defaulting for the application_id field of requests

CHANGELOG_BEGIN
- [Ledger API]: as a convenience when using JWT authentication, a
  request's application_id field can be left empty to ask the server to
  set the field to the id of the authenticated user or the application-id
  in the custom token. There are three services that currently specify
  application_id's in the request: CommandService,
  CommandSubmissionService, and CommandCompletionService.
CHANGELOG_END
2021-12-14 17:57:09 +00:00
Hubert Slojewski
259be4340b
kvutils: Move TransactionUtils from pkvutils [KVL-1166] (#12138)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-14 16:10:56 +00:00
Hubert Slojewski
83bb84a73f
kvutils: Change NodeId from String to a value class (#11943) 2021-12-14 13:01:57 +00:00
Sofia Faro
798137964b
Add requires keyword for syntax highlighting (#12113)
Part of #11978

changelog_begin
changelog_end
2021-12-14 12:20:41 +00:00
Sofia Faro
d1fb5e442c
interfaces: Implement 'requires' syntax (#12105)
* interfaces: Implement 'requires' syntax

part of: https://github.com/digital-asset/daml/issues/11978

ghc counterpart: https://github.com/digital-asset/ghc/pull/92

changelog_begin
changelog_end

* .

* ...

* ...

* ....

* ....

* update bazel-haskell-deps

* use requires syntax in tests

* .......

* .......

* update bazel-haskell-deps

* update InterfaceDesugared

* .........

* update bazel-haskell-deps

* ...........

* update bazel-haskell-depos

* update compile.yml

* update bazel-haskell-deps
2021-12-14 12:19:52 +00:00
Moritz Kiefer
01364f2863
Enable Daml Script user management tests (#12130)
With #12063 merged, these tests now pass \o/

changelog_begin
changelog_end
2021-12-14 11:57:41 +00:00
Simon Meier
75863c392f
Fix #12129: allow users' access to their own info (#12132)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-14 11:15:17 +00:00
Moritz Kiefer
f146bc814b
Escape daml-lf tracelog messages (#12116)
* Escape daml-lf tracelog messages

Currently veracode complains because this allows for clrf
injection (injecting newlines to make user input look like separate
log statements).

With this change
```
debug "abc"
debug "eaiu\neaiu"
debug "def"
debugRaw "abc
```

is logged as

```
[DA.Internal.Prelude:555]: \"abc\"
[DA.Internal.Prelude:555]: \"eaiu\neaiu\"
[DA.Internal.Prelude:555]: \"def\"
[DA.Internal.Prelude:555]: abc
```

You can debate whether we should escape the quotes are necessary but
90% of the reason why people add them is because they call `debug` on
strings when they should be using `debugRaw` so this seems fine to me.

changelog_begin
changelog_end

* fix tests

changelog_begin
changelog_end
2021-12-14 11:20:33 +01:00
Simon Meier
6c7e33cafe
Add Ledger API feature descriptor for user management. (#12095)
* Add Ledger API feature descriptor for user management.

Includes:
- enabling the UserManagementServiceIT tests by default; and skipping them for ledgers that do not support user management. The correct working of this is tested by the cross-version compatibility checks between the ledger-api-test-tool
- some refactoring wrt the parsing and representation of features the the ledger-api-test tool.

CHANGELOG_BEGIN
- [Ledger API]: introduce a feature descriptor for whether the ledger supports user management
CHANGELOG_END
2021-12-14 11:00:29 +01:00
Andreas Lochbihler
9a931854b2
add docs for command dedup (#11928)
CHANGELOG_BEGIN
Documentation on the new command deduplication mechanism
CHANGELOG_END
2021-12-14 09:36:32 +01:00
fabiotudone-da
f07596cb38
KV: VALIDATION_ERROR is not an internal error as it can also be caused by pruned contracts [KVL-1207] (#12108)
* KV: `VALIDATION_ERROR` is not an internal error anymore as it can also be caused by pruned contracts

CHANGELOG_BEGIN
- [Integration Kit] "VALIDATION_ERROR" is not an internal error anymore as it can also be caused by pruned contracts
CHANGELOG_END

* Fix v1 test

* Document errors produced due to lookup failures

* fmt

* Reformat comments
2021-12-13 18:26:39 +01:00
Simon Meier
f223528bfb
Implement PoC of user management for Ledger API server (fix #12014) (#12063)
CHANGELOG_BEGIN
- [User Management]: add support for managing participant node users and authenticating
  requests as these users using standard JWT tokens.
CHANGELOG_END

Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
Co-authored-by: Adriaan Moors <90182053+adriaanm-da@users.noreply.github.com>
2021-12-13 17:58:30 +01:00
Stefano Baghino
787dccb3d5
Bump TriggerServiceTest timeout (#12127)
changelog_begin
changelog_end
2021-12-13 14:44:40 +00:00
Bernhard Elsner
78a0677fb1
Fix command in triggers docs (#12123)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-13 15:35:33 +01:00