Commit Graph

130 Commits

Author SHA1 Message Date
nickchapman-da
5a7451bdf4
[engine] choice authority: clarify remaining things to do (#16752) 2023-04-27 18:30:32 +00:00
nickchapman-da
859d69b1a5
[engine] make authorizers field explicit in Node.Exercise (#16541)
* make authorizers field explicit in TX Node.Exercise

* use Option for exe-node choiceAuthorizers field, to be explicit about when old/default behaviour occurs
2023-04-06 13:05:44 +01:00
Remy
39d816f657
[Speedy] move compilation of disclosure outside of the Machine. (#16453)
Machine is not supposed to compile anything.
2023-03-06 19:04:31 +01:00
mziolekda
d8d55ca412
upgrade to scala 2.13.10 (#16423)
* update build system to scala 2.13.10

* fix failing scala files

* format

* fix compatibility

* resolve conflict on maven_install_2.13.json
2023-02-28 23:21:53 +00:00
Remy
df43a69020
[LF] Refactor ProcessedDisclosedContract (#16227)
- move it to com.daml.lf.transaction as it is an output
- include create node
2023-02-10 15:11:22 +01:00
Remy
aad8725ea7
[damlc] change default output to 1.15 (#16157) 2023-02-06 08:58:47 +01:00
Moisés Ackerman
8784ebd9ce
Add choiceController and choiceObserver functions (#16088)
* Add daml test file for choice functions

* Add HasChoice{Controller,Observer} type classes

* Add choice_{controller,observer} in LF

* Update GHC_REV

* Generate HasChoice{Controller,Observer} instances

* Remove manual instances in daml test file for choice functions

* Remove/forbid choice{Controller,Observer} before featureChoiceFuncs

* update *.desugared-daml golden files

* Apply code review suggestion

* Update GHC_REV (da-master-8.8.1)

Co-authored-by: Remy <remy.haemmerle@daml.com>
2023-01-18 14:50:00 +00:00
nickchapman-da
f9139f6761
revert actings_as_consortium changes; obsoleted by withAuthorityOf (#16067) 2023-01-16 15:27:56 +00:00
Carl Pulley
980464a866
Disable visibility checks for local contract keys (#15131)
Main part of  #14199

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2023-01-13 15:16:33 +00:00
tudor-da
3b87836fa7
Pass agreementText in ProcessedDisclosedContract (#15935)
changelog_begin
changelog_end
2022-12-20 15:42:04 +00:00
Remy
4135041a38
[Engine] compute agreement as part of contract caching (#15901) 2022-12-15 11:29:06 +00:00
nickchapman-da
2df734e1ce
[engine] consortium primitives in LF (#15864) 2022-12-14 15:06:15 +00:00
Remy
311f419842
LF: get rid of agreement text in ContractInst (#15818) 2022-12-14 10:39:29 +00:00
tudor-da
b4b1f6c7d9
Engine outputs interpreted disclosed contract metadata [DPP-1323] (#15751)
* Engine outputs interpreted disclosed contract metadata

* Addressed Remy's review comments
2022-12-04 20:11:53 +01:00
Matthias Schmalz
44dc483387
Revised test cases for authorization and added security annotations. (#15759)
CHANGELOG_BEGIN
CHANGELOG_END
2022-12-01 15:14:54 +00:00
Matthias Schmalz
eb59830a7b
Added a flag checkAuthorization to EngineConfig that allows clients to disable the authorization checks (for security testing). (#15748)
Generalized `object CheckAuthorization` to `trait AuthorizationChecker` which now gets injected into `PartialTransaction`.
2022-12-01 12:05:10 +00:00
dylant-da
05d9ee89f3
Improve interpretation error consistency (#15680)
* Add custom viewtype mismatch error

* Modify error message on EViewTypeMismatch to be like GHC error

* Make EUnknownMethodInInterfaceInstance error consistent w/ GHC

* Add MethodTypeMismatch error, sync with GHC error

* Prefix fields in new errors

* Intercept type mismatch in method to throw EMethodTypeMismatch

* Synchronize EUnknownField with GHC-side message

* Complete synchronizing EUnknownField with GHC-side error message

* satisfy lint and changelog check

CHANGELOG_BEGIN
CHANGELOG_END

* lint

* Fix TypingSpec for new EMethodTypeMismatch

* lint

* lint again

* EFieldTypeMismatch error for ERecUpd/ERecProj mismatch, like in GHC

* lint
2022-11-29 13:02:46 +00:00
Claudio Bley
e766f91338
Generate CSV file from all test evidences for ledger client components (#15372)
* Add new security categories

* Use test-security framework for OAuth2 middleware

* Regenerate `security-evidence.md`

* Explicitely exit the test-evidence generator

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Add //ledger-service/http-json:integration-tests-lib to evidence generator

* Skip maven artefacts on the classpath

The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
	at scala.collection.immutable.List.map(List.scala:250)
```

* Regenerate `security-evidence.md`

* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Print warning when a test suite could not be loaded

* Fix typo

* Use test-security framework for HTTP JSON

* Read files in tests lazily

The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for trigger service

* Use structural type to call `in` and `ignore` for different classes

* Remove Authentication category from EvidenceSecurity tool

There are no TEST_EVIDENCE annotations anymore.

* Add required trigger-runner-lib


* Import `scala.language.reflectiveCalls` where it is needed

* Remove left-over comments

* Add `test_evidence_binary` scala binary rule

This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.

* Process test suites, add transitive deps

* Support nested tests_suites in `test_evidence_binary`

* Remove debug print's

* Add missing dependencies to test-evidence:generator

* Abort if test suites cannot be loaded

* Cleanup

* Reinstate scalacopts in http-json

* Reword the test description to not drop information

* Fix typo

* Explicitly exit the JVM on exceptions

This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.

* Format test-evidence/BUILD.bazel

* Resolve file paths lazily

This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.

* Document new Security properties

* Print target directory and file name

* Clarify test descriptions

* Replace duplicate Security properties

Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-11-29 12:35:22 +01:00
Remy
462044bd93
[LF] add language version to scala StablePackage list (#15640)
fixes #11371
2022-11-28 10:42:21 +01:00
Remy
0bba409d0e
[LF] Make ApiCommand.Exercise work directly with type constructor (#15360)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-26 18:13:38 +00:00
Stephen Compall
7da769f37f
test number of query results after overlapping contract IDs in database (#15316)
CHANGELOG_BEGIN
CHANGELOG_END

* add single-party test case as well
2022-10-26 10:13:33 -04:00
Stephen Compall
7e7a8f2f2e
use domain.Party instead of String in json-api tests (#15329)
* use domain.Party instead of String in jwtForParties, headersWithPartyAuth

CHANGELOG_BEGIN
CHANGELOG_END

* use domain.Party instead of String in headersWithPartyAuth

* fix sharedAccountCreateCommand and some corners
2022-10-25 11:31:26 -04:00
Nicu Reut
f550b69463
Reorganize the metrics package structure [PLEN-29] (#15312) 2022-10-25 10:05:40 +00:00
fayi-da
7762d2a376
Removed unnecessary log message (#15323)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-24 14:35:18 +01:00
Chun Lok Ling
6e7c408484
Replace ContractTypeId with ContractTypeId.Template where appropriate (#15199)
* domain.ContractTypeId.Template
* ResolveKeyType to use .Template
* CreateAndExerciseCommand to use .Template
* keyTypeMap
* bitraverse

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-10-19 19:39:19 +00:00
Remy
4318a2341b
[LF] Cleanup StablePackage object (#15250)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-19 08:06:49 +00:00
Carl Pulley
ef692ef76e
Ensure trigger test DAR models are built for multiple LF versions
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-13 18:14:18 +01:00
Claudio Bley
e24497635d
Rework evidence security categories (#15226)
* Rework evidence security categories

using the 7 industry standard security testing areas:

* Authentication
* Authorization
* Confidentiality
* Integrity
* Availability
* Non-Repudiation
* Resilience

CHANGELOG_BEGIN
CHANGELOG_END

* Replace category `Input Validation`

* Replace category `Privacy`

* Replace category `Semantics`

* Replace category `Semantics`

* Replace category `Performance`

* Skip categories `NonRepudiation` and `Resilience` for now

There are currently no tests for these categories.

* Regenerate `security-evidence.md`
2022-10-13 09:55:45 +00:00
Stephen Compall
951817aee1
run query-store-independent tests less often (#15201)
* declare the separate kind of duplication

CHANGELOG_BEGIN
CHANGELOG_END

* move completely independent tests

* move many tests to 'QueryStoreIndependent'

* fix metering-report test to work with user tokens

* rename query-store/auth-dependent tests

- to QueryStoreAndAuthDependentIntegrationTest
- from AbstractHttpServiceIntegrationTestTokenIndependent

* security evidence moved
2022-10-11 09:27:07 -04:00
Nicu Reut
2f453a145c
Extract the testing metrics instance [PLEN-4] (#15192) 2022-10-10 18:52:49 +00:00
Carl Pulley
6364deb68e
Disable visibility checks for disclosed contracts and disclosed contract keys
* Fixes #14168

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-30 09:38:08 +01:00
Stephen Compall
9f1033a6ec
remove some domain.TemplateId usages (#15097)
* replace several TemplateId with ContractTypeId
* replace several TemplateId with ContractTypeId.Template
* deprecate, even though this misses so many usages
* link to #15098 for followups

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-27 07:53:10 -04:00
dylant-da
db652ef270
Improve EExpectedViewType (#15048)
* Add dylant-da as a codeowner on compiler-related directories

CHANGELOG_BEGIN
CHANGELOG_END

* SPlit EExpectedViewType into three variants with non-Text bodies

* Duplicate Haskell viewtype typechecking behaviour onto Scala side

* Split TypingSpec tests for different viewtype failures

* lint & changelog

CHANGELOG_BEGIN
CHANGELOG_END

* Fix lint
2022-09-26 14:38:12 +00:00
Remy
6dd9a45e50
[Engine] small cleaning of LookupError (#15060)
* [Engine] small cleaning of LookupError

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-23 13:46:14 +00:00
mziolekda
41dc32a016
Remove reliance on implicit party allocation in CI tests [DPP-1234] (#15037)
* Remove reliance on implict party allocation in the authorization tests of the sandbox-on-x

CHANGELOG_BEGIN
CHANGELOG_END

* Remove the HOCON and cli config parameters

* Remove implicit party allocation from the Submission Service

* Remove support for implicit parties from the bridge

* Fix codegen reliance on implicit party allocation

* Fix daml script tests

* Fix trigger tests

* format security-evidence.md post rebase

* Fix json-api tests

* One more json-api test

* Fix haskell tests

* Corrections post-review

* Cosmetic improvements

* Fix race condition in party allocation handling
2022-09-22 10:20:53 +02:00
Stephen Compall
77672d2896
better JsNull check for Oracle; test scenarios for nested cases (#15022)
* a couple comparison query scenarios with nested position
* same JSON null test, but different
- 'gt string' (scenario 0) fails without this change
- 'gt int' (scenario 1) succeeds regardless of this change

CHANGELOG_BEGIN
- [JSON API with Oracle] Fix some nested queries that were returning no
  data.  DisableContractPayloadIndexing must be set to true to take
  advantage of this fix.
CHANGELOG_END
2022-09-21 10:21:35 +00:00
Stephen Compall
a9ebecfe5c
retest Oracle with payload indexing (#15041)
* point to #14844 and #15040 for some TODOs
* duplicate the Oracle tests, disable large literals when indexing is on
* split test file for parallelization

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-20 10:32:58 -04:00
Stephen Compall
36e49c737b
factor stream-starting code in websocket JSON API tests (#15008)
CHANGELOG_BEGIN
CHANGELOG_END
2022-09-13 19:26:59 +00:00
Stephen Compall
5bec0ba4d2
remove redundant StatusCode checks from many tests (#15007)
* ensure Status is redundant for every parseResponse user
* make parseResponse test and discard the HTTP status
* use resultContractId more

CHANGELOG_BEGIN
CHANGELOG_END
2022-09-13 09:40:49 -04:00
Chun Lok Ling
419bf9b040
Fire a warning if both config file and cli opts found. (#14981)
* fire a warning if both config file and cli opts found.

* add change log

CHANGELOG_BEGIN
CHANGELOG_END

* fixes test case

* fmt
2022-09-08 21:22:25 +01:00
Chun Lok Ling
32583edb7d
v1/stream/query interface subscription support (#14919)
* replace Set[TemplateId.Resolved] with ResolvedQuery in StreamPredicate val resolved
CHANGELOG_BEGIN
CHANGELOG_END

* make it compile first and fix those TODOs iteratively.

* `Unsupported or ResolvedQuery monoid` for collectively combine `ResolvedQuery`s; handle unsupported cases

* cleanup todo

* cleanup

* use resolvedQuery instead and add change log

CHANGELOG_BEGIN
CHANGELOG_END

* compile

CHANGELOG_BEGIN
CHANGELOG_END

* added test cases

CHANGELOG_BEGIN
CHANGELOG_END

* Address Ray's comment

CHANGELOG_BEGIN
Users can now subscribe `v1/stream/query` with queries with single interface id.
CHANGELOG_END

* Address Ray's comment

* Address Ray's comment

* fmt

* fix error message

* fix error message

* Addressed Stephen's comment

* Addressed Stephen's comment

* Addressed Stephen's comment
2022-09-07 09:20:23 +01:00
Raymond Roestenburg
fbe3d94bfe
14818 fetch by contract id with interface (#14917)
* Fallback to in-memory for interface ID.

CHANGELOG_BEGIN
- [JSON API] /v1/fetch by interface ID
CHANGELOG_END

* Added fix: not setting key in ActiveContract in interface case, fix for https://github.com/digital-asset/daml/pull/14841#discussion_r961960344
2022-09-06 11:03:31 +02:00
Raymond Roestenburg
d13695c75e
Draft /v1/query with interface ID, without query store (#14841)
Support /v1/query by interfaceID or a non-empty set of template ID's, fallback to in-memory for an interfaceId query.

Co-authored-by: Stephen Compall <stephen.compall@daml.com>


CHANGELOG_BEGIN
- [JSON API] /v1/query with a non-empty set of template ID’s or one interface ID
CHANGELOG_END
2022-09-02 12:49:06 +02:00
Remy
5a07b6596f
LF: check viewtype is a monomorphic record (#14802)
CHANGELOG_BEGIN
CHANGELOG_END
2022-08-25 10:02:13 +02:00
Simon Maxen
937f8d1d45
Switch metering report to using Struct [DPP-1135] (#14767)
* Switch metering report to using Struct

changelog_begin
changelog_end
2022-08-22 08:55:06 +01:00
Remy
48abfeb9dc
LF: distinguish template Id from interface Id in exercise command (#14755)
preliminary work for #14747

CHANGELOG_BEGIN
CHANGELOG_END
2022-08-19 14:00:40 +02:00
Remy
18ef46052f
LF: force view evaluation during by_interface updates (#14738)
Possible errors during such evaluation are uncatchable

CHANGELOG_BEGIN
CHANGELOG_END
2022-08-18 16:49:29 +00:00
Remy
4ac224b518
LF: Move TemplateOrInterface form language package to data package (#14748)
to be able to use it without direct dependence to language package.

CHANGELOG_BEGIN
CHANGELOG_END
2022-08-18 16:20:18 +02:00
Remy
b53c7c3799
LF test use consistently record for interface view. (#14746)
CHANGELOG_BEGIN
CHANGELOG_END
2022-08-18 10:27:43 +00:00
Carl Pulley
26b48133b4
Test expected behaviour when superfluous (i.e. unused) contracts are disclosed in commands (#14617)
* Resolves #14350

CHANGELOG_BEGIN

* Engine/speedy-level tests for explicit disclosure (#14227): Test expected behaviour when superfluous (i.e. unused) contracts are disclosed in commands.

CHANGELOG_END
2022-08-16 14:15:47 +01:00