Commit Graph

8744 Commits

Author SHA1 Message Date
Robert Autenrieth
d922a562a8
DPP-752 refactor event strategy (#12008)
* Reset string interning between tests

* Add StorageBackendEvents tests

* Extend LedgerDao ACS tests

* Remove dead code

changelog_begin
changelog_end

* Refactor EventStrategy

changelog_begin
changelog_end

* Remove dead code

* Refactor filterPartiesClause

* Avoid materializing a collection

* remove interning from arrayIntersectionNonEmptyClause

* Fix internedAllParties

* Add test for filtering witnesses

* Handle empty readers

* Add note about resetting string interning state

* Throw exceptions on unexpected empty input
2021-12-21 09:32:26 +01:00
Moisés Ackerman
ba0c6c9841
Set --enable-scenarios to False by default (#12156)
* Set --enable-scenarios to False by default

changelog_begin
changelog_end

* Enable scenarios for shake test

* Add --enable-scenarios=yes for //compiler/lsp-tests

* Add enable_scenarios arg to rules_daml/daml.bzl helpers

* daml_compile
* daml_build_test
* daml_test

* Add enable_scenarios to ledger/test-common helper da_scala_dar_resources_library

* Add --enable-scenarios=yes for //ledger/test-common

* Remove unused scenario in //ledger/test-common PingPong test

* Add --enable-scenarios=yes for //daml-lf/tests

* Add --enable-scenarios=yes for //language-support/java tests

* Add --enable-scenarios=yes for group-chat example

* Add --enable-scenarios for //ledger/sandbox-perf LargeTransaction test

* Add --enable-scenarios=yes for //docs tests

* Add --enable-scenarios=yes for //daml-lf/scenario-interpreter tests

* format bazel

* Add --enable-scenario=yes for 'daml_doc_test's

* Add --enable-scenarios=yes for DamlDocTestIntegration

* Add --enable-scenarios=yes for DamlcTest

* Add --enable-scenarios=yes for Test/DataDependencies

* Add --enable-scenarios=yes for daml-ghc-deterministic.sh

* Add --enable-scenarios=yes for Test.IncrementalBuilds

* Add --enable-scenarios=yes for tests:memory-bond-trading and tests:memory-examples

* Add --enable-scenarios=yes for daml-assistant/integratio-tests
2021-12-20 19:31:06 +01:00
Hubert Slojewski
56baf036b7
Rename kv-transaction-support library [KVL-1168] (#12212)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-20 16:33:36 +01:00
nicu-da
5b142eba63
[ledger-api-test-tool] - use offset support feature descriptor [kvl-1218] (#12211) 2021-12-20 06:53:39 -08:00
Moritz Kiefer
0142c6a34e
Add forM_ compat definition (#12209)
I keep confusing myself because of this and I expect I’m not the only
one. We have this for forM already so this only seems natural.

The definition is in DA.Foldable rather than DA.Internal.Compatible
since it doesn’t make sense to have it imported when you don’t import forA_.

changelog_begin
changelog_end
2021-12-20 13:45:28 +00:00
Moritz Kiefer
99f9776dd4
New split release for docs tests (#12186)
Release using the split release process so I can test the docs changes
I made.

changelog_begin
changelog_end
2021-12-20 13:44:04 +00:00
nicu-da
e182fc5026
[ledger-api] - Move deduplication duration validation into the KV WriteService (#12151)
changelog_begin
ledger-api - Remove validation for duration deduplication periods, which was validating that the deduplication duration is smaller than the max deduplication duration. This is now up to the ledger to enforce.
changelog_end
2021-12-20 05:33:46 -08:00
nicu-da
53d55784a4
[ledger-api] - Add command deduplication features as a feature descriptor [KVL-1218] (#12181)
changelog_begin
changelog_end
2021-12-20 04:33:58 -08:00
Moritz Kiefer
040f1a933f
Release attempt over 9000 (#12206)
https://www.youtube.com/watch?v=dQw4w9WgXcQ

changelog_begin
changelog_end
2021-12-20 11:23:00 +00:00
Moritz Kiefer
606a8ef08e
Drop code to upload to com.digitalasset (#12203)
We didn’t have any artifacts there since SDK 1.0 and we’re not
planning to add any. We suspect that the permission denied errors
might come from the fact that we need a different profile id for that
and while we don’t use the repo to upload anything we still create the
staging repo so this might fix that issue.

changelog_begin
changelog_end
2021-12-20 10:16:53 +01:00
Moritz Kiefer
6d6a91189a
Fix target name of non-sphinx tarball (#12205)
We reference that in copy-unix-release-artifacts.sh so the name
actually matters.

changelog_begin
changelog_end
2021-12-20 08:50:38 +00:00
Moritz Kiefer
f057765cd2
Snapshot release attempt (#12201)
I don’t even know which attempt this is anymore.

changelog_begin
changelog_end
2021-12-18 13:29:48 +00:00
Victor Peter Rouven Müller
c5de61ae9e
[JSON-API] Also accept user tokens as auth source (#12075)
* WIP

* Remove the dummy implementation and replace it with an actual working implementation

* Make it compile!

* Add working tests for the user management support in the json api

CHANGELOG_BEGIN

- [JSON-API] Added basic support for the new user management feature of the ledger such that user tokens are now accepted instead of the legacy tokens

CHANGELOG_END

* Simplify the create iou test case and adjust the test case name to be correct

* Add additional test that covers that the overwrite of actAs&readAs still works via the meta object

* Make it work with unauthenticated ledgers too

* Fix compile error & wrong behaviour & add test coverage for non auth ledgers

* Clean up the diff

* Address 66312e9940 (r770782884)

* Address 66312e9940 (r770750653)

* Addressing 66312e9940 (r770751958)

* Address 66312e9940 (r770736671)

* Address 66312e9940 (r770734395) and 66312e9940 (r770783237)

Co-authored-by: Stefano Baghino <stefano.baghino@digitalasset.com>
2021-12-18 12:54:12 +00:00
azure-pipelines[bot]
9b3645fb26
update NOTICES file (#12199)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-18 13:09:00 +01:00
Moritz Kiefer
5de5eb06d8
Update Maven profile id (#12200)
Apparently the profile id changed with the switch to the new host.

changelog_begin
changelog_end
2021-12-18 12:18:12 +01:00
nicu-da
453fd64d05
[ledger-api-test-tool] - Add ability to disable tests based on ledger features [kvl-1218] (#12185)
changelog_begin
changelog_end
2021-12-17 12:15:38 -08:00
Stefano Baghino
2a71b69cef
Minor fixes to issue templates (#12195)
1. Using Markdown headers for sections instead of bold font
2. Removing myself as a default assignee -- I am already subscribed to the firehose and there is nothing to gain for anyone from this

changelog_begin
changelog_end
2021-12-17 17:07:32 +00:00
Moritz Kiefer
0c9683183c
Support user management in standalone trigger runner (#12190)
* Support user management in standalone trigger runner

This PR adds a `--ledger-user` option to the trigger runner which runs
the trigger as the primary party of the user and with all readAs
claims available to the user.

fixes #12025

changelog_begin

- [Daml Trigger] `daml trigger` now accepts a `--ledger-user` option
  which can be used instead of specifying the primary party and all
  other parties the user has claims to as ``-ledger-party` and
  ``-ledger-readas`.

changelog_end

* .

changelog_begin
changelog_end
2021-12-17 16:45:16 +00:00
Simon Maxen
51b9405f5c
TransactionNodesStatistics.stats to return case class of same name (#12159)
* TransactionNodesStatistics.stats now returns case class of same name

changelog_begin
changelog_end

* Rename TransactionNodesStats to Detail and make inner class

* Rename TransactionNodesStatistics.stats to apply

* Rename TransactionNodesStatistics to TransactionNodeStatistics

* Reformat

* Remove rollback from statistic action count

* Update with review comments

* Update with review comments
2021-12-17 15:27:53 +00:00
Hubert Slojewski
0a2c207e93
kvutils: Create our Identifier message and remove references to Daml-LF one [KVL-1167] (#12189)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-17 15:54:56 +01:00
Stefano Baghino
52110e31a3
Remove Extractor (#12188)
* Remove Extractor

Extractor is being removed after a long time being in Labs status.

This should improve the flakiness on CI.

🔥

changelog_begin
Extractor has been removed from the SDK
changelog_end

* Remove Extractor documentation

* Remove Extractor from CODEOWNERS

* Remove Extractor references in Daml-LF build files

* Remove Extractor references in the Daml SDK assistant

* Remove Extractor from the SDK

* Remove Extractor reference from CONTRIBUTING.md
2021-12-17 13:43:23 +00:00
Hubert Slojewski
c4b5de609a
kvutils: Remove references to Daml-LF Archive from kvutils protos [KVL-1167] (#12168)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-17 13:24:46 +01:00
Moritz Kiefer
1329fde631
Release another 2.0 snapshot (#12182)
This time with the new maven host.

changelog_begin
changelog_end
2021-12-17 11:42:25 +00:00
Moritz Kiefer
5f688dd704
Cache localContract calls in contractKeyInputs (#12183)
changelog_begin
changelog_end
2021-12-17 11:37:24 +00:00
Moritz Kiefer
adb1f0e716
Split up docs postprocessing & release in split-release (#12176)
* Split up docs postprocessing & release in split-release

Follow up to #12172

The buld in the separate assembly should now only have to run the
sphinx-build step and then merge it with the non sphinx sources and
run lualatex on the pdf sources with the pdf fonts which is hopefully
simple enough that we can easily replicate it.

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-12-17 11:22:16 +01:00
nicu-da
2ba3d60947
Add conversions for CompletionStage to Future which unwraps failures (#12165)
changelog_begin
changelog_end
2021-12-17 09:34:20 +00:00
Moritz Kiefer
d230cc88f6
Switch to new sonatype host (#12179)
They migrated our account so hopefully this should work without any
other changes and fix our publishing issues.

I’m keeping the long timeout for now since I don’t know what an
appropriate timeout is.

changelog_begin
changelog_end
2021-12-17 09:19:02 +01:00
tudor-da
ea7d49d4d9
Implement Sandbox-on-X runner [DPP-823] (#12150)
* Implement Sandbox-on-X runner

CHANGELOG_BEGIN
CHANGELOG_END

* Address Marton's comments

* Extracted AkkaSubmissionsBridge

* Addressed second round of review comments
2021-12-17 07:38:16 +00:00
Sergey Kisel
4c8be783b2
use domain.Offset instead of String in ContractsFetch functions (#12160)
* Initial commit

CHANGELOG_BEGIN
CHANGELOG_END

* formatting fix

* fix up graph DSL whitespace

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2021-12-16 18:14:39 +00:00
Stefano Baghino
42adfdc857
Get rid of deprecated components (#12167)
* Get rid of deprecated components

Removes:
- mentions of the Node.js bindings
- mentions of the Scala bindings
- usage of the Scala codegen as an SDK tool
- Java bindings "reactive components"

changelog_begin
The Node.js bindings are no longer supported as part of the Daml SDK
The Scala bindings are no longer supported as part of the Daml SDK
The Java bindings reactive components are no longer supported as part of the Daml SDK
The `daml codegen` command has dropped support for Scala
changelog_end

* Fix typo noticed by @cocreature

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Address https://github.com/digital-asset/daml/pull/12167#issuecomment-995714093

* Remove quickstart-scala from the SDK tarballs

* Fix failing codegen test

* Address https://github.com/digital-asset/daml/pull/12167#pullrequestreview-833991243

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-12-16 16:55:19 +00:00
Remy
7ecc12645e
LF: make Struct lookup logarithmic (#12170)
CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 16:48:45 +00:00
akshayshirahatti-da
4745768ad4
[Oauth2-Middleware] Changes to introduce HOCON+pureconfig for oauth2-middleware (#12061)
* Changes to introduce HOCON+pureconfig for oauth2-middleware

CHANGELOG_BEGIN
CHANGELOG_END

* remove Cli arg parsing code + cleanup based on code review

* addition of a minimal config and changes to README.md

* keep existing cli args, but load from config file if provided

* fix broken docs build

* make tests OS independent

* Fail/error on supplying both config file and cli opts for startup, address code review comments
2021-12-16 16:36:32 +00:00
Sofia Faro
e52469c1dc
Add a test that mkMethod call causes a type error. (#12175)
Relevant discussion: https://github.com/digital-asset/daml/pull/12171#discussion_r770553009

changelog_begin
changelog_end
2021-12-16 15:52:44 +00:00
Moritz Kiefer
97e3a1a557
Refactor docs build (#12172)
* Refactor docs build

This is in preparation for the split docs build where we publish our
sphinx sources here and then build the final docs combined with the
Canton docs in the assembly repo.

Apologies for the messy diff. The idea here is the following:

1. We bundle together all files used as sphinx inputs in
sphinx-source-tree. This will be what we publish.
2. We factor out the calls to sphinx-build for the pdf and html
build to be as generic as possible and share the code.
3. The existing rules for the html build & pdf docs don’t invoke
sphinx anymore and instead only keep the postprocessing (copying other
html files or calling lualatex).

In a followup PR I’ll also simplify the postprocessing steps and then
finally publish those artifacts as part of our build.

changelog_begin
changelog_end

* fmt

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2021-12-16 15:37:21 +00:00
Sofia Faro
8b05a533ce
Refactor daml start options, add true/false to yes/no/auto flags. (#12169)
* Refactor daml start & add true/false to yesNoAuto.

- Refactor the way we pass arguments to daml start. We were relying on
  positional arguments with newtypes, but this is super cumbersome. I
  changed it to a RecordWildCards-style approach, where we don't need
  quite so many newtypes, and no more positional arguments.

- "--start-navigator" flag had some custom logic to accept "true" and
  "false". I don't see why we can't just accept "true" and "false"
  anywhere we use the "yes/no/auto" flags, so I just changed that and
  got rid of the custom logic.

- The way "auto" was handled for this flag was incorrect, since "auto"
  is supposed to be equivalent the default, i.e. not passing any
  flag. I changed it so auto is equivalent to not passing an argument.
  (I.e. it looks in daml.yaml for the start-navigator option).

changelog_begin
changelog_end

* dont pass in shutdownStdinClose to runStart
2021-12-16 14:40:36 +00:00
Robin Krom
1ed02369eb
interfaces: fix mkMethod erasure (#12171)
We erase the call to `mkMethod` in `convertExpr` instead of pattern
matching against the binding. This fixes a bug when the method body
contained statements using typeclass dictionaries such as
`do [2] === [2]`.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 15:22:46 +01:00
fabiotudone-da
2940a7b5c2
[ledger-api-test-tool] - existing_submission_id and completion_offset assertions for ALREADY_EXISTS rejections in command dedup ledger API tests [KVL-1020] (#12140)
changelog_begin
changelog_end
2021-12-16 06:11:39 -08:00
Moritz Kiefer
98d4e177cf
Release 2.0 snapshot (#12164)
This time with the increased Maven timeout which is hopefully enough
to get it to succeed.

changelog_begin
changelog_end
2021-12-16 12:21:48 +00:00
Remy
39ee326a51
LF: Drop wrong toArraySeq method from ImmArray. (#12163)
The method is not correctly implement, as it should slice the
underlying array before returning it. Since it is not possible to
implement it in constant time, we simply drop it, in favor in other
conversion.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 10:18:52 +00:00
Moritz Kiefer
101356ba36
Bump sonatype timeout to absurdity (#12162)
changelog_begin
changelog_end
2021-12-16 09:38:03 +00:00
Simon Meier
64e228b72b
integration-kit: remove all its docs (#12157)
CHANGELOG_BEGIN
- [Daml Integration Kit]: removed from Daml 2.0
CHANGELOG_END
2021-12-16 10:30:19 +01:00
azure-pipelines[bot]
1189dd5915
update NOTICES file (#12161)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
2021-12-16 08:54:29 +00:00
Marton Nagy
115fbba799
Introducing DbSupport, pulling up DbDispatcher building in the factory hierarchies (#12152)
This PR prepares the landscape for seamless integration of Index DB with non IndexService related
front components like UserManagementStore.
DbSupport is a simply a DbDispatcher and a StorageBackendFactory. Creation of it comes also in Flyway migrated flavor for capturing common need.
This is a pure refactoring PR: no behavior/feature changes

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-16 08:49:20 +01:00
Samir Talwar
9d00861b4e
resources-akka: Wait for the bounded source queue to finish. (#12142)
* resources-akka: Wait for the bounded source queue to finish.

Otherwise, we may get submissions after dependencies have shut down.

CHANGELOG_BEGIN
CHANGELOG_END

* resources-akka: Simplify the interface and use clearer type param names.
2021-12-15 20:42:06 +00:00
Brian Healey
bb89f369c0
revert to use synopsys detect 7.1.0 (#12155)
* revert to use synopsys detect 7.1.0 run-full-compat: true

CHANGELOG_BEGIN
CHANGELOG_END

* run on branch run full compat

run-full-compat: true

* run on branch run full compat

run-full-compat: true

* revert changes to run branch
2021-12-15 14:39:28 -05:00
Simon Meier
51df74490b
user management: complete authentication tests (#12143)
* user management: complete authentication tests (#12062)

Also incorporate naming suggestion wrt 'createUserByAdmin' from @mziolekda

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-15 14:40:05 +00:00
Robin Krom
5af58a1652
interfaces: doc generation for interface methods (#12096)
This adds documentation generation for interface choices and methods.

CHANGELOG_BEGIN
CHANGELOG_END
2021-12-15 13:57:09 +00:00
Moritz Kiefer
506eee16f5
Extend VSCode extension troubleshooting section (#12154)
I’ve had to look this up for the second time today so documenting it
seems very sensible.

changelog_begin
changelog_end
2021-12-15 13:56:30 +00:00
Moisés Ackerman
149474f3cb
Drop scenarios from the user-facing SDK (#12128)
* Add --enable-scenarios hidden flag
* Fail LFConversion when a scenario is defined and --enable-scenarios=no
* Remove unused optIsGenerated=True test cases
* Refactor getIntegrationTests
* Add no-scenarios-enabled tests
* Add enable-scenarios flag to scenario-service server
* Add test ensuring that scenario-returning functions are still allowed
* Remove 'enable scripts' option (--daml-script)
* Add --scripts synonym to --scenarios
* Rephrase scenarios flags descriptions

changelog_begin
changelog_end
2021-12-15 13:39:14 +01:00
azure-pipelines[bot]
9a1fa5ab10
release 2.0.0-snapshot.20211214.8692.0.63940872 (#12147)
* release 2.0.0-snapshot.20211214.8692.0.63940872

This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@cocreature is in charge of this release.

Commit log:
```
6394087280 Add transaction statistics to CompletionInfo (#12139)
a03df88f7c DPP-721 Remove old ACS implementation (#12045)
81990bc8b3 Fix #12133: correct error for tokens with invalid user-ids (#12134)
717772850d Update Logback to 1.2.8 (#12141)
8c1173c8f1 User management application id defaulting (#12100)
259be4340b kvutils: Move `TransactionUtils` from `pkvutils` [KVL-1166] (#12138)
83bb84a73f kvutils: Change `NodeId` from `String` to a value class (#11943)
798137964b Add requires keyword for syntax highlighting (#12113)
d1fb5e442c interfaces: Implement 'requires' syntax (#12105)
01364f2863 Enable Daml Script user management tests (#12130)
75863c392f Fix #12129: allow users' access to their own info (#12132)
f146bc814b Escape daml-lf tracelog messages (#12116)
6c7e33cafe Add Ledger API feature descriptor for user management. (#12095)
9a931854b2 add docs for command dedup (#11928)
f07596cb38 KV: `VALIDATION_ERROR` is not an internal error as it can also be caused by pruned contracts [KVL-1207] (#12108)
f223528bfb Implement PoC of user management for Ledger API server (fix #12014) (#12063)
787dccb3d5 Bump TriggerServiceTest timeout (#12127)
78a0677fb1 Fix command in triggers docs (#12123)
77a37cf213 Drop damlc code for LF <= 1.11 (#12121)
caa8072d91 Drop unused primitives for interface contract id conversion (#12114)
82465e2e85 Drop Decimal completely from damlc (#12120)
e1a2da0771 [ledger-api] - Add deduplication_offset as a valid deduplication period for commands [KVL-1172] (#12041)
9ff8851488 Fix a docs display issue with Safari and Firefox (#12035)
36106c39de Drop featureNumeric (#12119)
4cc88a2fc3 `ledger-api-bench-tool` - item rate SLO [DPP-790] (#12033)
7f76cf1bdf Drop featureAnyType and featureTypeRep (#12112)
c5708c96c5 es: mitigate log4j vuln (#12118)
b6b6ecd669 repair ES cluster (#12117)
937849b101 Define asciiToLower and asciiToUpper in DA.Text (#12109)
124f8854f0 daml-docs: Refactor context type (#12111)
778def6f48 Drop non-type synonym code from LF conversion (#12110)
25ae69eb01 [DPP-657] Make error codes docs nicer (#12079)
8c87a0e327 LF: Factorize interpretation loop emulation spread in multiple tests. (#12102)
9f8e640b1a Use ExecutionContext.parasitic instead of DirectExecutionContext. (#11783)
9a01bb9b86 Fix filters in ExceptionRaceConditionIT (#12106)
3fae3b2e3e Stop flyway from printing sensitive jdbc string at info level (#12088)
ae8c2788b8 Split release test with cross-platform damlc (#12107)
8011d602c8 Support multiple participant in daml script’s user management (#12091)
1989a2def9 LF: Check non circularity of `requires` Interface field
```
Changelog:
```
- [Ledger API]: as a convenience when using JWT authentication, a
  request's application_id field can be left empty to ask the server to
  set the field to the id of the authenticated user or the application-id
  in the custom token. There are three services that currently specify
  application_id's in the request: CommandService,
  CommandSubmissionService, and CommandCompletionService.
- [Ledger API]: introduce a feature descriptor for whether the ledger supports user management
Documentation on the new command deduplication mechanism
- [Integration Kit] "VALIDATION_ERROR" is not an internal error anymore as it can also be caused by pruned contracts
- [User Management]: add support for managing participant node users and authenticating
  requests as these users using standard JWT tokens.
[ledger-api] - Add `deduplication_offset` as a valid type for the command deduplication period
- [Integration Kit] - A minimum count rate objective can be defined for a stream in the ledger-api-bench-tool
```

CHANGELOG_BEGIN
CHANGELOG_END

* Update LATEST

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-12-15 12:06:00 +01:00