Commit Graph

152 Commits

Author SHA1 Message Date
Paul Brauner
d3521956b8
remove redundant comment (#17828)
* remove redundant comment

* update security evidence
2023-11-14 18:57:42 +00:00
Samuel Williams
6d7543d84c
Add ensure clause check whenever contractInfo is needed (#17822)
* Add ensure clause check whenever contractInfo is needed

* Update comment

* Avoid double precondition on create

* Update evaluation order test with new precondition calls

* Update security evidence
2023-11-14 16:16:21 +00:00
Paul Brauner
890dee12bf
Tie right-to-left evaluation order to LF v2 (#17811)
* Tie right-to-left evaluation order to LF v2

* fix typo
2023-11-14 10:34:31 +01:00
Marton Nagy
78cc750714
Scala upgrade to 2.13.11 (#17803) 2023-11-13 09:05:39 +01:00
Remy
6ef062dac9
Revert "Nmarton/pekko rebased (#17789)" (#17796)
This reverts commit d310692d1b.

@remyhaemmerle-da messed up and merged the PR too fast.
2023-11-09 21:23:18 +00:00
Marton Nagy
d310692d1b
Nmarton/pekko rebased (#17789)
* Scala upgrade to 2.13.11
* Switch from akka to pekko
* Also disable gatling based libs to start from a clean stage
2023-11-09 18:17:33 +01:00
Simon Maxen
ae5e9feda9
Allow for the construction of shared GlobalKeys.
* Revert "Add SharedGlobalKey"

This reverts commit 6592cb7bd5.

* Adding shared flag to GlobalKey

* Remove shared flag from GlobalKey

* Support old and new grcp error parsing

* Add tests for hashContractKey
2023-11-06 20:00:38 +00:00
Remy
9575b72a01
[Engine] allow (up/down)grade in create command (#17451)
This PR provides a basic API for ledger client to create contracts with down/up-grade support.
One can send a create Command where the payload contains

-  (upgrade path) less fields than requires by the package definition (as long as the missing fields are optional), in such case the engine assume those fields are None
-  (downgrade path) more fields than requires by the package definition (additional field should be node), in such case the engine drop those fields.

Here are some restrictions we assume for this first step:

 - For the sake of simplicity, we forbid field reordering when upgrade is enable
 - The package of explicit disclosed contract is required
 - we ignore package ID complelty inside value (probably not what we want in final version)
2023-10-19 18:02:02 +00:00
Paul Brauner
f4ee00361e
Make LF v2 non backwards-compatible with LF v1 in the compiler (#17492)
* initial commit

* split gen-stable-packages-v_i into two targets

* rename compatibleWith to canDependOn

* update damlc integration tests annotation to allow for 2.x

* use the right package ID for preconditionFailed when compiling to V2

* fuse stable-packages-v1 and stable-packages-v2 into one single filegroup

* Fix DA.Daml.LF.Ast.Tests

* remove leftover 'undefined' in Version.hs

* progress on fixing DataDependencies.hs

* fix Cross-SDK typeclasses test for 2.dev

* Fix the 'Typeclasses and instances' data dependency test

* Update comment

* fix //compiler/damlc/tests:packaging

* Add TODO

* parameterize the machine by the langage version, hardcode v1 in script v2, v2 in script v2, v1 in exports

* get EngineTests to pass

* fix more tests

* fix canton integration tests

* formatting

* fix more tests

* fix transactionversiontest

* fix exceptiontest

* Fix ValueEnricherSpec

* Fix EngineInfoTest

* fix PartialTransactionSpec

* fix upgragetest

* fix TransactionSnapshot

* Fix ContractKeySpec

* Fix ReinterpretTest

* fix InterfaceViewSpec

* fix InterfacesTest

* fix stable package v1 names

* fix validate.sh tests

* formatting

* Fix ChoiceAuthorityTest

* fix explicit disclosure test

* Fix SpeedyTest

* formatting

* Fix integration test

* fix data dependency tests

* fix package vetting count, increased due to metadata being added

* Redact stable package IDs in error messages in order for the ExceptionSemantics test to work for both v1 and v2

* cleanup

* fix Daml2ScriptTestRunner

* fix JsonApiIT and daml-script golden tests

* fix daml3-script runner test

* enable v2 for all integration tests

* formatting

* fix NodeSeedsTest

* fix since-lf annotations

* add comments, improve consistency

* stop hardcoding V1 in runPureExpr and runPureSExpr

* formatting

* remove harcoding of LFv1 in ConcurrentCompiledPackages.apply

* Parameterize Compiler.Config.Default with major language version

* remove global parser implicit and default package ID and language version
2023-10-11 11:40:17 +02:00
Remy
176c8505b9
[Speedy] Move back contract cache in speedy (#17523)
For reference, this undoes some of the changes from #17171.
2023-10-06 10:04:09 +02:00
Paul Brauner
a5cd445d3b
Introduce LF2 major version (#17348)
- Introduces a new major version, "2", in the daml_lf proto
- Adds new major versions to the compiler and the engine
- Updates all code that assumes only one major version
- Updates all code that assumes only one dev version
2023-09-13 10:01:42 +02:00
Paul Brauner
7585f72d7a
Add flag-protected right-to-left evaluation (#17253)
Right-to-left evaluation allows for a faster and simpler interpreter, but is incompatible with the semantics of Daml 2.x. Adding it behind a flag so we can switch to it in Daml 3.0.

See PR #17253 for ample details.
2023-08-15 13:10:44 +00:00
nickchapman-da
4b60e35a5d
[engine] Restructure contract caching (#17171) 2023-08-09 16:09:13 +00:00
nickchapman-da
9fddcb956c
Thread enableContractUpgrading through engine. (#17139) 2023-07-20 12:54:45 +00:00
Remy
e04dfb5fa1
[LF] cleanup Result#consume (#17125) 2023-07-13 17:12:02 +02:00
nickchapman-da
8a3a7d20f9
[engine] improve speedy record value representation (#17054) 2023-06-30 15:25:50 +00:00
Remy
8e1bfb31f8
remove dead engine option (#17017)
forgotten in #12464
2023-06-19 15:18:45 +02:00
Remy
dd98cbec76
[LF] Cleanup API for disclosed contract (#16920) 2023-05-31 14:39:53 +02:00
Simon Maxen
761071f215
Tree transaction building (#16912)
* With test transaction building refactor
2023-05-26 14:03:09 +01:00
Remy
912e9a9f33
[LF] make generic Dev interpretation error (#16859) 2023-05-18 13:01:43 +00:00
Matthias Schmalz
6d82c6bf00
Removing securityTestLayer from test evidence (#16843)
The securityTestLayer property used to be defined on a per test suite basis. The value may vary from test case to test case. Therefore, it is not always possible to assign a correct value. Removing the property to resolve this problem.

CHANGELOG_BEGIN
CHANGELOG_END
2023-05-10 10:04:49 +00:00
nickchapman-da
3b68f14e74
[engine] choice authority eval order testing (#16835)
* extend eval order testing for choice authority

* update security evidence
2023-05-10 09:36:32 +01:00
nickchapman-da
5a7451bdf4
[engine] choice authority: clarify remaining things to do (#16752) 2023-04-27 18:30:32 +00:00
nickchapman-da
859d69b1a5
[engine] make authorizers field explicit in Node.Exercise (#16541)
* make authorizers field explicit in TX Node.Exercise

* use Option for exe-node choiceAuthorizers field, to be explicit about when old/default behaviour occurs
2023-04-06 13:05:44 +01:00
Remy
39d816f657
[Speedy] move compilation of disclosure outside of the Machine. (#16453)
Machine is not supposed to compile anything.
2023-03-06 19:04:31 +01:00
mziolekda
d8d55ca412
upgrade to scala 2.13.10 (#16423)
* update build system to scala 2.13.10

* fix failing scala files

* format

* fix compatibility

* resolve conflict on maven_install_2.13.json
2023-02-28 23:21:53 +00:00
Remy
df43a69020
[LF] Refactor ProcessedDisclosedContract (#16227)
- move it to com.daml.lf.transaction as it is an output
- include create node
2023-02-10 15:11:22 +01:00
Remy
aad8725ea7
[damlc] change default output to 1.15 (#16157) 2023-02-06 08:58:47 +01:00
Moisés Ackerman
8784ebd9ce
Add choiceController and choiceObserver functions (#16088)
* Add daml test file for choice functions

* Add HasChoice{Controller,Observer} type classes

* Add choice_{controller,observer} in LF

* Update GHC_REV

* Generate HasChoice{Controller,Observer} instances

* Remove manual instances in daml test file for choice functions

* Remove/forbid choice{Controller,Observer} before featureChoiceFuncs

* update *.desugared-daml golden files

* Apply code review suggestion

* Update GHC_REV (da-master-8.8.1)

Co-authored-by: Remy <remy.haemmerle@daml.com>
2023-01-18 14:50:00 +00:00
nickchapman-da
f9139f6761
revert actings_as_consortium changes; obsoleted by withAuthorityOf (#16067) 2023-01-16 15:27:56 +00:00
Carl Pulley
980464a866
Disable visibility checks for local contract keys (#15131)
Main part of  #14199

Co-authored-by: Remy Haemmerle <Remy.Haemmerle@daml.com>
2023-01-13 15:16:33 +00:00
tudor-da
3b87836fa7
Pass agreementText in ProcessedDisclosedContract (#15935)
changelog_begin
changelog_end
2022-12-20 15:42:04 +00:00
Remy
4135041a38
[Engine] compute agreement as part of contract caching (#15901) 2022-12-15 11:29:06 +00:00
nickchapman-da
2df734e1ce
[engine] consortium primitives in LF (#15864) 2022-12-14 15:06:15 +00:00
Remy
311f419842
LF: get rid of agreement text in ContractInst (#15818) 2022-12-14 10:39:29 +00:00
tudor-da
b4b1f6c7d9
Engine outputs interpreted disclosed contract metadata [DPP-1323] (#15751)
* Engine outputs interpreted disclosed contract metadata

* Addressed Remy's review comments
2022-12-04 20:11:53 +01:00
Matthias Schmalz
44dc483387
Revised test cases for authorization and added security annotations. (#15759)
CHANGELOG_BEGIN
CHANGELOG_END
2022-12-01 15:14:54 +00:00
Matthias Schmalz
eb59830a7b
Added a flag checkAuthorization to EngineConfig that allows clients to disable the authorization checks (for security testing). (#15748)
Generalized `object CheckAuthorization` to `trait AuthorizationChecker` which now gets injected into `PartialTransaction`.
2022-12-01 12:05:10 +00:00
dylant-da
05d9ee89f3
Improve interpretation error consistency (#15680)
* Add custom viewtype mismatch error

* Modify error message on EViewTypeMismatch to be like GHC error

* Make EUnknownMethodInInterfaceInstance error consistent w/ GHC

* Add MethodTypeMismatch error, sync with GHC error

* Prefix fields in new errors

* Intercept type mismatch in method to throw EMethodTypeMismatch

* Synchronize EUnknownField with GHC-side message

* Complete synchronizing EUnknownField with GHC-side error message

* satisfy lint and changelog check

CHANGELOG_BEGIN
CHANGELOG_END

* lint

* Fix TypingSpec for new EMethodTypeMismatch

* lint

* lint again

* EFieldTypeMismatch error for ERecUpd/ERecProj mismatch, like in GHC

* lint
2022-11-29 13:02:46 +00:00
Claudio Bley
e766f91338
Generate CSV file from all test evidences for ledger client components (#15372)
* Add new security categories

* Use test-security framework for OAuth2 middleware

* Regenerate `security-evidence.md`

* Explicitely exit the test-evidence generator

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Add //ledger-service/http-json:integration-tests-lib to evidence generator

* Skip maven artefacts on the classpath

The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
	at scala.collection.immutable.List.map(List.scala:250)
```

* Regenerate `security-evidence.md`

* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Print warning when a test suite could not be loaded

* Fix typo

* Use test-security framework for HTTP JSON

* Read files in tests lazily

The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for trigger service

* Use structural type to call `in` and `ignore` for different classes

* Remove Authentication category from EvidenceSecurity tool

There are no TEST_EVIDENCE annotations anymore.

* Add required trigger-runner-lib


* Import `scala.language.reflectiveCalls` where it is needed

* Remove left-over comments

* Add `test_evidence_binary` scala binary rule

This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.

* Process test suites, add transitive deps

* Support nested tests_suites in `test_evidence_binary`

* Remove debug print's

* Add missing dependencies to test-evidence:generator

* Abort if test suites cannot be loaded

* Cleanup

* Reinstate scalacopts in http-json

* Reword the test description to not drop information

* Fix typo

* Explicitly exit the JVM on exceptions

This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.

* Format test-evidence/BUILD.bazel

* Resolve file paths lazily

This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.

* Document new Security properties

* Print target directory and file name

* Clarify test descriptions

* Replace duplicate Security properties

Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-11-29 12:35:22 +01:00
Remy
462044bd93
[LF] add language version to scala StablePackage list (#15640)
fixes #11371
2022-11-28 10:42:21 +01:00
Remy
0bba409d0e
[LF] Make ApiCommand.Exercise work directly with type constructor (#15360)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-26 18:13:38 +00:00
Stephen Compall
7da769f37f
test number of query results after overlapping contract IDs in database (#15316)
CHANGELOG_BEGIN
CHANGELOG_END

* add single-party test case as well
2022-10-26 10:13:33 -04:00
Stephen Compall
7e7a8f2f2e
use domain.Party instead of String in json-api tests (#15329)
* use domain.Party instead of String in jwtForParties, headersWithPartyAuth

CHANGELOG_BEGIN
CHANGELOG_END

* use domain.Party instead of String in headersWithPartyAuth

* fix sharedAccountCreateCommand and some corners
2022-10-25 11:31:26 -04:00
Nicu Reut
f550b69463
Reorganize the metrics package structure [PLEN-29] (#15312) 2022-10-25 10:05:40 +00:00
fayi-da
7762d2a376
Removed unnecessary log message (#15323)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-24 14:35:18 +01:00
Chun Lok Ling
6e7c408484
Replace ContractTypeId with ContractTypeId.Template where appropriate (#15199)
* domain.ContractTypeId.Template
* ResolveKeyType to use .Template
* CreateAndExerciseCommand to use .Template
* keyTypeMap
* bitraverse

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-10-19 19:39:19 +00:00
Remy
4318a2341b
[LF] Cleanup StablePackage object (#15250)
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-19 08:06:49 +00:00
Carl Pulley
ef692ef76e
Ensure trigger test DAR models are built for multiple LF versions
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-13 18:14:18 +01:00
Claudio Bley
e24497635d
Rework evidence security categories (#15226)
* Rework evidence security categories

using the 7 industry standard security testing areas:

* Authentication
* Authorization
* Confidentiality
* Integrity
* Availability
* Non-Repudiation
* Resilience

CHANGELOG_BEGIN
CHANGELOG_END

* Replace category `Input Validation`

* Replace category `Privacy`

* Replace category `Semantics`

* Replace category `Semantics`

* Replace category `Performance`

* Skip categories `NonRepudiation` and `Resilience` for now

There are currently no tests for these categories.

* Regenerate `security-evidence.md`
2022-10-13 09:55:45 +00:00