Commit Graph

146 Commits

Author SHA1 Message Date
mziolekda
34743570bd
Rewire the build to use ledger-api within canton (#17959)
* Rewire the build to use ledger-api within canton

* same for canton-3

* fix buf

* fix buf again

* remove references to canton from canton-3x
2023-12-04 16:11:13 +01:00
mziolekda
8e0b71bb15
clean up test common (#17846)
* move stuff around inside test-common

* move timeout stuff to lapitt, where it is used

* move logging stuff to ledger/error

* move StreamConsumer to libs-scala/grpc-test-utils

* remove dead usages of test-common

* move SimpleHttpServer to a dedicated lib

* format

* final cleanup of the test-common

* add new libraries to maven
2023-11-15 22:36:03 +00:00
mziolekda
b81ee69abe
disentangle ledger api (#17831)
* make testing-utils independent of ledger-api

* remove dead files

* move non ledger-api stuff to libs-scala
2023-11-14 22:44:10 +01:00
Marton Nagy
6933514df5
Switch from Akka to Pekko (#17814)
* Automated renames by bash script

This commit is exclusively contains changes by the bash script.
For the bash script is present at the pull request.

* Manual pekko migration changes

* adapt fully qualified name references
* adapt pekko package declarations
* adapt bazel files with dependency changes
* adapt canton pekko lib shade_rule
* adapt logger configuration declarations
* pin maven dependencies
* revert incorrect changes by script to compatibility module

Workarounds for further TODOs:
* disable http-json-perf and libs-scala/gatling-utils modules to maintain clean pekko dependencies (without akka)
* disable GraphQLSchemaSpec test (sangria library needs to be upgraded)

* Formatting
2023-11-13 23:22:22 +00:00
Remy
6ef062dac9
Revert "Nmarton/pekko rebased (#17789)" (#17796)
This reverts commit d310692d1b.

@remyhaemmerle-da messed up and merged the PR too fast.
2023-11-09 21:23:18 +00:00
Marton Nagy
d310692d1b
Nmarton/pekko rebased (#17789)
* Scala upgrade to 2.13.11
* Switch from akka to pekko
* Also disable gatling based libs to start from a clean stage
2023-11-09 18:17:33 +01:00
Simon Maxen
ba34290e23
Inject language version lookup into gRPC error parsing (#17767) 2023-11-08 14:49:53 +00:00
Simon Maxen
ae5e9feda9
Allow for the construction of shared GlobalKeys.
* Revert "Add SharedGlobalKey"

This reverts commit 6592cb7bd5.

* Adding shared flag to GlobalKey

* Remove shared flag from GlobalKey

* Support old and new grcp error parsing

* Add tests for hashContractKey
2023-11-06 20:00:38 +00:00
Simon Maxen
6592cb7bd5
Add SharedGlobalKey (#17650)
* Add SharedGlobalKey

* Update with review comments

* Add temporary canton compatibility
2023-10-26 14:31:36 +01:00
Andreas Triantafyllos
44c0c51bb3
Refactor ledger/error (#17509) 2023-09-29 17:51:32 +02:00
Samuel Williams
aecf70c900
Implement TrySubmit in daml3-script (#17043)
Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2023-07-25 18:22:31 +02:00
Samuel Williams
f82fb77102
Update tests to match canton changes (#17093) 2023-07-12 16:53:32 +01:00
mziolekda
06abd5c649
Clean up ledger dependencies (#16894)
* remove sandbox-on-x project

update bazel readme

update release artifacts

comment out last remaining SoX test

remove ledger-runner-common

remove participant-state-kv-errors

remove recovering-indexer-integration-tests

remove participant-integration-api

update doc pages

cleanup ledger-api-auth usage

remove participant-state

fix build

fix build

clean up ledger-api-common part I

clean up ledger-api-comon part II

clean up ledger-api-common part III

remove ledger/metrics

clean up ledger-api-health and ledger-api-domain

* remove ledger-configuration ad ledger-offset

* remove ledger-grpc and clean up participant-local-store

* reshuffle few more classes

* format
2023-05-23 13:44:17 +00:00
mziolekda
95cc249ddd
Remove sandbox on x (#16890)
* remove sandbox-on-x project

* update bazel readme

* update release artifacts

* comment out last remaining SoX test

* remove ledger-runner-common

* remove participant-state-kv-errors

* remove recovering-indexer-integration-tests

* remove participant-integration-api

* update doc pages

* cleanup ledger-api-auth usage

* remove participant-state

* fix build

* fix build
2023-05-23 09:25:54 +02:00
mziolekda
1a3980bf89
Separate test-common from the rest of the ledger code (#16307)
* move files

* format

* wrong file

* fix conformance tests

* add readme

* fix compatibility tests

* fix haskell tests
2023-02-14 15:29:13 +00:00
Gary Verhaegen
151e12b81a
bump copyright (#16002)
This is the result of:

- Updating `./COPY` to say `2023`.
- Running `./dev-env/bin/dade-copyright-headers update .`
2023-01-04 18:21:15 +01:00
Sergey Kisel
e880f26b36
Identity Provider Management exposed via API [DPP-1334] (#15774) 2022-12-09 10:25:14 +01:00
Claudio Bley
e766f91338
Generate CSV file from all test evidences for ledger client components (#15372)
* Add new security categories

* Use test-security framework for OAuth2 middleware

* Regenerate `security-evidence.md`

* Explicitely exit the test-evidence generator

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Add //ledger-service/http-json:integration-tests-lib to evidence generator

* Skip maven artefacts on the classpath

The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
	at scala.collection.immutable.List.map(List.scala:250)
```

* Regenerate `security-evidence.md`

* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Print warning when a test suite could not be loaded

* Fix typo

* Use test-security framework for HTTP JSON

* Read files in tests lazily

The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for trigger service

* Use structural type to call `in` and `ignore` for different classes

* Remove Authentication category from EvidenceSecurity tool

There are no TEST_EVIDENCE annotations anymore.

* Add required trigger-runner-lib


* Import `scala.language.reflectiveCalls` where it is needed

* Remove left-over comments

* Add `test_evidence_binary` scala binary rule

This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.

* Process test suites, add transitive deps

* Support nested tests_suites in `test_evidence_binary`

* Remove debug print's

* Add missing dependencies to test-evidence:generator

* Abort if test suites cannot be loaded

* Cleanup

* Reinstate scalacopts in http-json

* Reword the test description to not drop information

* Fix typo

* Explicitly exit the JVM on exceptions

This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.

* Format test-evidence/BUILD.bazel

* Resolve file paths lazily

This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.

* Document new Security properties

* Print target directory and file name

* Clarify test descriptions

* Replace duplicate Security properties

Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-11-29 12:35:22 +01:00
mziolekda
a02c30440a
remove deprecated warnings everywhere [DPP-1281] (#15661)
* remove deprecated warnings in /ledger

* update generated documentation
2022-11-24 12:05:08 +00:00
Ratko Veprek
32b45966ad
Unified error category malicious or faulty with security alert (#15649)
Before this change, we had two error categories that were meant for the same purpose: malicious or faulty and security alert. Now, they have been merged together. As malicious or faulty was misunderstood and sometimes used instead of "SystemInternalAssumptionViolation", we've dropped that term.

Related to canton 10981
2022-11-22 14:05:06 +01:00
mziolekda
60ab087e71
Fix copy pasta in ErrorCategory (#15543)
* Fix copy pasta in ErrorCategory

* update documentation
2022-11-14 08:39:57 +01:00
tudor-da
346bb48904
Extract Ledger API error definitions in //ledger/ledger-api-errors [DPP-1269] (#15439)
* Extract Ledger API error definitions in //ledger/error-definitions

changelog_begin
SERVICE_INTERNAL_ERROR error code is created in CommonErrors and replaces the usage of LEDGER_API_INTERNAL_ERROR in ledger-api-client (as a fix).
changelog_end

* Tests in //ledger/error do not depend on //ledger/ledger-api-errors

* Remove unused dependency from //ledger/error

changelog_begin
changelog_end

* Address Martino's comment
2022-11-07 17:24:43 +01:00
pbatko-da
7652944a9c
Improve wording of the conveyance message in error codes docs (#15043)
changelog_begin
changelog_end
2022-09-28 08:55:12 +00:00
pbatko-da
ca81adeb3e
6 - Ledger API changes for UM/UP extensions for Hub [DPP-1211] (#14937)
changelog_begin

[Ledger API Specification]:
1. Add 'is_deactivated' attribute to participant users.
2. Introduce participant server's local metadata for parties and participant users consisting of:
 - a resource version for optional concurrent change control,
 - modifiable key-value based annotations.
3. Add experimental update RPCs for parties and participant users:
  - participant user's modifiable fields are: 'primary_party', 'is_deactivated', 'metadata.annotations',
  - party's modifiable fields are 'local_metadata.annotations'.
4. Discourage use of 'party_details.display_name' in favor of using party's metadata annotations.

changelog_end


Other notes:

- no-op updates are allowed,
- update paths are resource relative,
- as annotation deletion is by providing an key with the empty string value,
- as a corollary, only non-empty strings are valid annotation values.
2022-09-26 12:56:55 +00:00
Carl Pulley
aa062cde42
Final step of explicit disclosure cached contract refactor
* Fixes #13897

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-21 18:12:55 +01:00
tudor-da
62c64c6e2a
Correct error message for not found templates/interfaces in stream subscriptions [DPP-1220] (#14935)
* Correct error message for not found templates/interfaces in stream subscription

changelog_begin
When subscribing to Ledger API transaction streams with unknown template or interface ids
filter, a client is served with a NOT_FOUND/TEMPLATES_OR_INTERFACES_NOT_FOUND error code.
changelog_end

* Temporarily create a custom preview and dev conformance test for Canton
in order to allow the changed test exclusion
2022-09-07 16:17:32 +02:00
Matthias Schmalz
08559be6af
New error category: SecurityAlert (#14957)
CHANGELOG_BEGIN
Added new error category "SecurityAlert".
Corrected documentation of GRPC status codes for error categories MaliciousOrFaultyBehavior, AuthInterceptorInvalidAuthenticationCredentials, and InsufficientPermission.
CHANGELOG_END
2022-09-07 15:57:04 +02:00
tudor-da
d4646d7f9d
[ED] Explicit disclosure conformance tests [DPP-1095] (#14352)
* Disclosed contracts conformance tests

changelog_begin
changelog_end

* Addressed review comments

* Addressed Moritz' review comments

* Add conformance test asserting disabled feature

* More exhaustive testing:
* Aserting metadata contents
* Test duplicate keys/contract ids
* Test local key lookup visibility

* Addressed Carrie's review comments

* Added two more tests:
* Archival race test
* Superfluous inconsistent contracts race test

* Update ledger/ledger-api-tests/suites/src/main/scala/com/daml/ledger/api/testtool/suites/v1_dev/ExplicitDisclosureIT.scala

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>

Co-authored-by: mziolekda <marcin.ziolek@digitalasset.com>
2022-09-05 08:49:21 +00:00
Matthias Schmalz
7728d61565
Ignore non-singleton subtypes of ErrorCode when creating error code documentation. (#14829)
CHANGELOG_BEGIN
CHANGELOG_END
2022-08-25 14:59:42 +02:00
Simon Maxen
38925252fb
Add support for metering report signing using community and enterprise keys [DPP-1136] (#14735)
* Add support for metering report signing using community and enterprise keys

changelog_begin
changelog_end
2022-08-18 11:34:12 +01:00
Sergey Kisel
80c8ff4ce6
Interface view enrichment logic, LF-Value translation changes [DPP-1158] (#14731)
* Interface view enrichment logic, LF-Value translation changes [DPP-1158]

CHANGELOG_BEGIN
CHANGELOG_END
2022-08-18 10:06:04 +02:00
Carl Pulley
19a2b986bc
Disclosure testing invalid contract type
* Test that invalidly typed contracts are rejected

CHANGLELOG_BEGIN

* Engine/speedy-level tests for explicit disclosure (#14227): test that invalidly typed contracts are rejected.

CHANGELOG_END
2022-08-09 18:45:44 +01:00
Remy
abd47e972b
[Speedy] Do not crash when encountering an inconsistent contract key (#14276)
but rather return a proper interpretation error.

CHANGELOG_BEGIN
CHANGELOG_END
2022-07-05 10:28:38 +00:00
Simon Maxen
5db3cc4bd5
Rate limit streams [DPP-1001] (#14166)
Add support for stream based rate limiting

changelog_begin
changelog_end
2022-07-04 14:28:46 +01:00
Robin Krom
36b39aca7b
Move disclosed contract logic to speedy (#14288)
* Move disclosed contract logic to speedy

This moves the disclosed contract logic from the engine to speedy.

Fixes #14090.

* back to versioned contract instances

CHANGELOG_BEGIN
CHANGELOG_END
2022-07-01 08:40:10 +00:00
Simon Maxen
6a1e3afb71
Switch rate limiting to using self service error codes [DPP-1000] (#14216)
* Switch rate limiting to using self service error codes

changelog_begin
Rate limiting updated to use self service error codes.
changelog_end
2022-06-21 16:49:22 +01:00
Remy
2d5094160e
[Engine] clean key uniqueness mode (#14194)
This is a follow up of #14175 and #14176.  

Basically
-  we kill the `On` mode which is not used anymore.
- we factorize the visibility check.
- we rename `LocalContractNotVisible` into `ContractNotVisible`

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-16 12:04:45 +00:00
Hubert Slojewski
281de9ac2e
Publish daml-lf encoder, parser and error-test-lib [KVL-1424] (#13730) 2022-04-29 13:59:19 +02:00
Nicu Reut
a792de39a5
Update scalafmt to 3.5.1. (#13584)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-28 09:57:04 +00:00
Hubert Slojewski
35b68962e8
Remove remaining kvutils dependencies [KVL-1245] (#13695)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 13:42:18 +02:00
Robin Krom
fcd3b6622b
ifaces:support exercising by required interface (#13554)
* ifaces:support exercising by required interface

This adds support to exercise an interface choice on a contract ID,
where the interface is required by one of the implemented interfaces of
the contract template.

Fixes #13434.

CHANGELOG_BEGIN
CHANGELOG_END

* some improvements based on review

* added a test plus a bugfix

* Update daml-lf/interpreter/src/main/scala/com/digitalasset/daml/lf/speedy/PhaseOne.scala

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>

* 2 more interface tests for the command preprocessor

Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
2022-04-12 18:15:29 +00:00
Matthias Schmalz
c21a135d4f
Changed log level of PARTICIPANT_BACKPRESSURE to INFO. (#13555)
(Otherwise, an excessive rate of incoming commands will flood the log file with backpressure warnings.)

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-12 09:49:55 +02:00
davidrichards-da
ef55141788
Added clarity around security risk errors (#13407)
* Adding section to error codes page and updating descriptions

CHANGELOG_BEGIN
CHANGELOG_END
Added a section in the error codes page and improved description to errors

* Removing the multi-line descriptions

* Changing the added sentence

swapping 'due to security reasons' 'for security reasons'

* Updated formatting

Ran the fmt.sh script to give me the correct formatting.

* Updating the generated error codes

CHANGELOG_BEGIN
CHANGELOG_END
Updating the generated error codes
2022-03-30 15:50:00 +01:00
pbatko-da
c6a0275f6e
[Error codes] Fill in missing contexts that were lost where reflection based context population was removed [DPP-606] (#13264)
And rename error arguments that haven't been in camel case yet.

changelog_begin
changelog_end
2022-03-16 15:07:24 +01:00
pbatko-da
b63fdc7031
[Error codes] Increase visibility of formatContextAsString (#13304)
changelog_begin
changelog_end
2022-03-16 13:26:57 +01:00
pbatko-da
e0965709fe
[Error codes] Remove .asGrpcStatusFromContext and .asGrpcErrorFromContext from BaseError trait [DPP-606] (#13262)
changelog_begin
changelog_end
2022-03-16 09:30:07 +01:00
pbatko-da
c7266dd60c
Remove ErrorCauseExport + some file renames [DPP-606] (#13259)
changelog_begin
changelog_end
2022-03-15 12:51:32 +01:00
pbatko-da
ee6a5f9e0e
Handle misc error codes todos. [DPP-606] (#13248)
changelog_begin
changelog_end
2022-03-14 17:16:31 +01:00
pbatko-da
e69343dfb3
[Error codes] Reorganize error groups file structure for better readability and easier modification. [DPP-606] (#13263)
changelog_begin
changelog_end
2022-03-14 15:13:26 +01:00
pbatko-da
92e3e9e2df
Add unit tests for ErrorCode [DPP-799] (#13236)
changelog_begin
changelog_end
2022-03-14 12:38:23 +01:00