Commit Graph

464 Commits

Author SHA1 Message Date
Carl Pulley
ef692ef76e
Ensure trigger test DAR models are built for multiple LF versions
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-13 18:14:18 +01:00
Claudio Bley
e24497635d
Rework evidence security categories (#15226)
* Rework evidence security categories

using the 7 industry standard security testing areas:

* Authentication
* Authorization
* Confidentiality
* Integrity
* Availability
* Non-Repudiation
* Resilience

CHANGELOG_BEGIN
CHANGELOG_END

* Replace category `Input Validation`

* Replace category `Privacy`

* Replace category `Semantics`

* Replace category `Semantics`

* Replace category `Performance`

* Skip categories `NonRepudiation` and `Resilience` for now

There are currently no tests for these categories.

* Regenerate `security-evidence.md`
2022-10-13 09:55:45 +00:00
Carl Pulley
c710081ed3
Log Runner stream runtime exceptions at ERROR and with a stack trace
CHANGELOG_BEGIN
CHANGELOG_END
2022-10-10 15:46:24 +01:00
nickchapman-da
3180094987
[engine] avoid construction of unrestricted speedy expression applications (#15079)
Avoid construction of unrestricted speedy expression applications.SEApp (alias for SEAppGeneral)

- Replace occurrences of SEApp(General) to use SEAppAtomic, with SELet1 as required
- Add new helper SEApp for many cases where arguments are simple values.
- Rename the most general unrestricted constructor as SEAppGeneral_DEPRECATED. The only remaining use is by fromUpdateSExpr and fromScenarioSExpr, due to our unprincipled approach to stack-trace. See TODO in pushLocation. I hope to fix this in a future PR.
- Rename the slightly less unrestricted constructor as SEAppOnlyFunIsAtomic_DEPRECATED. The only call site being in Anf.scala (for the case when ANF is not performed). Maybe one day we can perform ANF in all cases, and hence remove this one remaining usage.
2022-09-26 14:33:51 +01:00
pbatko-da
ca81adeb3e
6 - Ledger API changes for UM/UP extensions for Hub [DPP-1211] (#14937)
changelog_begin

[Ledger API Specification]:
1. Add 'is_deactivated' attribute to participant users.
2. Introduce participant server's local metadata for parties and participant users consisting of:
 - a resource version for optional concurrent change control,
 - modifiable key-value based annotations.
3. Add experimental update RPCs for parties and participant users:
  - participant user's modifiable fields are: 'primary_party', 'is_deactivated', 'metadata.annotations',
  - party's modifiable fields are 'local_metadata.annotations'.
4. Discourage use of 'party_details.display_name' in favor of using party's metadata annotations.

changelog_end


Other notes:

- no-op updates are allowed,
- update paths are resource relative,
- as annotation deletion is by providing an key with the empty string value,
- as a corollary, only non-empty strings are valid annotation values.
2022-09-26 12:56:55 +00:00
mziolekda
41dc32a016
Remove reliance on implicit party allocation in CI tests [DPP-1234] (#15037)
* Remove reliance on implict party allocation in the authorization tests of the sandbox-on-x

CHANGELOG_BEGIN
CHANGELOG_END

* Remove the HOCON and cli config parameters

* Remove implicit party allocation from the Submission Service

* Remove support for implicit parties from the bridge

* Fix codegen reliance on implicit party allocation

* Fix daml script tests

* Fix trigger tests

* format security-evidence.md post rebase

* Fix json-api tests

* One more json-api test

* Fix haskell tests

* Corrections post-review

* Cosmetic improvements

* Fix race condition in party allocation handling
2022-09-22 10:20:53 +02:00
Raymond Roestenburg
deba5b4630
Doc add live ready auth 14649 (#15045)
* Added readyz route.

* Doc for liveness and readiness endpoints for auth middleware.

CHANGELOG_BEGIN
- oauth2 middleware: liveness and readiness endpoints
CHANGELOG_END
2022-09-20 16:38:49 +02:00
Moisés Ackerman
009c3c306b
Add interface instance syntax, including retroactive interface instances (#14715)
* Extract qualifyLocally in LFConversion

* Update ghc rev

* Add parent type argument to interface instance desugaring types

* Check interface instance parent in LFConversion

* Convert interface instances in interface declarations

* Update daml-test-files to use 'interface instance' syntax

* Update other tests to use 'interface instance' syntax

* Update interface reference docs to use 'interface instance' syntax

* Update other files to use 'interface instance' syntax

* Add todo in DA.Daml.LF.Ast.Pretty

* Add daml-test-files for retroactive interface instances

changelog_begin
changelog_end
2022-08-17 12:06:59 +00:00
dylant-da
0d37134bc2
Change placeholder unit viewtypes to empty records (#14656)
* Update views in integration-v1dev to be EmptyInterfaceView instead of ()

* Update expected desugared daml for `EmptyInterfaceView`

* Update compiler Shake / DataDependencies viewtypes to EmptyInterfaceView

* Add record viewtypes to docs, update docs

* Add empty records to daml-lf, language-support, ledger, trigger tests

* Qualify EmptyInterfaceView in ledger semantic Interface tests

* Fix imports for EmptyInterfaceView in IIou / CIou / Transferrable

* empty changelog commit

CHANGELOG_BEGIN
CHANGELOG_END

* Qualify EmptyInterfaceView import

* Fix imports of EmptyInterfaceView in TypeScript codegen tests

* Accept Interface doc changes

* Add EmptyInterfaceView to DataDependencies tests

* Remove missing interface viewtype test - all interfaces must have views

* Replace all EmptyInterfaceView with EmptyInterfaceView {}
2022-08-10 16:45:23 +00:00
dylant-da
8e8e0dabdf
Convert viewtype and view methods to LF (#14456)
* Add DA_Internal_Interface to convertTypeDef exclusion check

* Move desugared types/values for interface views into DA.Internal.Desugar

* Convert viewtype and view method declarations

* Typecheck view types and view declarations

* Add unit viewtypes to InterfaceChoiceCollision tests

* Add unit viewtypes to more compiler/damlc tests

* Modify empty interfaces/implements to have viewtypes/views

* Add unit viewtypes to remaining Interface tests

* Test for error thrown when viewtype not specified

* Fix daml-lf/ interface tests to use new syntax / unit views

* Add placeholder view of type Unit to TestInterfaces

CHANGELOG_BEGIN
CHANGELOG_END

* Fix typescript interface tests with unit viewtype

* Add unit viewtype to Java & Scala codegen

* Add unit viewtype to triggers tests interface

* Add unit viewtypes to ledger/test-common

* Add unit viewtypes to ledger-service/http-json

* Fix some damlc tests

* Use viewtype syntax in InterfaceViewNonSerializable test

* Remove HasInferfaceView instances in convertBind

* Add unit view to QualifiedInterface.daml test

* Generate HasInterfaceView instances in DataDependencies

* Add unit viewtypes and views to compiler/damlc/tests

* Document reconstruction of HasInterfaceView

* Move desugared types/values for interface views into DA.Internal.Desugar

* Add desugarable function view w/ EViewInterface desugaring

* Remove templateName and viewtype from generated EViewInterface

* Update desugared-daml

* Proof of concept: use _view -> EViewInterface prim via HasInterfaceView

* Move view, HasInterfaceView to DA.Internal.Interface

* Update desugared-daml tests

* Stop removing and re-generating HasInterfaceView instances

* Do not generate docs for generated "_view_" methods

* Report errors with interface TypeConNames prettily

* Fix InterfaceViewNotSpecified @ERROR to reflect new error text

* Don't import magic `view` method from Prelude to avoid name clash

* Update shake interface goto definition test locations

* Temp disable view spec tests, will re-implement views in speedy

* Add unit viewtype to Java code snippet docs

* Update Interface syntax docs for views

* Add unit viewtype to Transferrable.daml

* Add back "empty" interface implementation to interfaces.rst

* Re-enable view tests by having _view methods on InterfaceViews.daml file

* Remove deprecated commented-out view desugaring code
2022-08-03 16:23:38 +01:00
nickchapman-da
bbfcc55af7
[engine] Transmit completed transaction via SResultFinalValue. (#14416)
* Transmit completed transaction via SResultFinalValue.

changelog_begin
changelog_end

* remove spurious mods

* mark two commented-out tests with TODO 14431

* address small comments

* rename: SResultFinalValue --> SResultFinal
2022-07-18 09:45:37 +00:00
Sergey Kisel
f9521f27eb
Accept new form of JWT tokens [DPP-1102] (#14323)
* Accept new form of jwt tokens [DPP-1102]

CHANGELOG_BEGIN
CHANGELOG_END
2022-07-04 17:41:08 +02:00
Remy
52ec65040f
[Engine] Rename values/arguments of type PackageInterface (#14305)
to avoid confusion with Daml interface.

CHANGELOG_BEGIN
CHANGELOG_END
2022-07-01 15:38:04 +02:00
Sergey Kisel
6a80a1a82f
Removal of sandbox-classic module and binary (#14201)
* Removal of sandbox-classic module and binary

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-21 11:09:51 +02:00
Sergey Kisel
f36252ff55
Removal of unused file daml-platform.sh. (#14125)
Does not seem to be used, as well as refers to deprecated sandbox-classic

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-08 16:04:42 +02:00
Sergey Kisel
1d8f195438
Sandbox Classic Removal in Tests (#13934)
CHANGELOG_BEGING
CHANGELOG_END
2022-06-07 19:54:52 +00:00
Remy
7638e086ae
Script: infer the typeId of Choice in daml (#13971)
part of #13653

goes with digital-asset/ghc#122

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-02 14:02:12 +00:00
Raymond Roestenburg
e3f01870a3
Added readyz endpoint. (#13902)
CHANGELOG_BEGIN
The ‘/readyz’ endpoint has been added to the trigger service to support readiness probes.
CHANGELOG_END
2022-05-19 08:34:26 +02:00
Remy
f4338bdece
Trigger: return less confusing message when trigger is not found. (#13904)
* Trigger: return less confusing message when trigger is not found.

fixes #13893

CHANGELOG_BEGIN
CHANGELOG_END
2022-05-18 09:31:16 +00:00
Moritz Kiefer
d3280ac87d
Add queryFilter to triggers (#13769)
* Add queryFilter to triggers

changelog_begin

- [Daml Triggers] Add `queryFilter` matching Daml Script’s
  `queryFilter` which queries contracts of a given template and filters
  them down to those where the predicate holds.

changelog_end

fixes #13746

* Update triggers/daml/Daml/Trigger.daml
2022-05-02 17:36:57 +00:00
Nicu Reut
a792de39a5
Update scalafmt to 3.5.1. (#13584)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-28 09:57:04 +00:00
Remy
79af226966
LF: Add a convenient companion object for ArrayList (#13684)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 12:36:36 +02:00
Stephen Compall
4a3d0b316b
use matchers for trigger test assertions (#13569)
* use shouldBe
- suggested by @cocreature

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-12 16:51:25 +00:00
Stephen Compall
f6fed6eaed
use matchers for trigger service test assertions (#13567)
* remove boolean usage of assertTriggerStatus
* remove CompatAssertion

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-11 19:07:17 +00:00
Stephen Compall
289e1c3692
use matchers for auth middleware test assertions (#13565)
* use OptionValues instead of Option#get

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-11 14:47:57 -04:00
Victor Peter Rouven Müller
ad4ed5c4b2
Add security evidence for auth middleware (#13424)
changelog_begin
changelog_end
2022-04-05 11:25:56 +02:00
Victor Peter Rouven Müller
dcd726e113
Add security evidence to the simple trigger tests (#13231)
changelog_begin
changelog_end
2022-03-22 17:01:35 +00:00
Stefano Baghino
739173bf84
Remove references to missing appender (#13294)
changelog_begin
[Trigger Service] Spurious logging warnings will not be printed when running `daml trigger-service`
[Oauth2 Middleware] Spurious logging warnings will not be printed when running `daml oauth2-middleware`
changelog_end

Removed references to a missing `stderr-appender` in several configuration file.

This removes warnings when running `daml trigger-service` and `daml oauth2-middleware`.

This will also remove those same warnings from several tests (HTTP JSON API and Trigger Service).
2022-03-15 18:02:02 +01:00
Stefano Baghino
0ffb88a6ca
Add logback.xml file in the distributable JAR for the Trigger Service (#13295)
Fixes #13042

changelog_begin
[Trigger Service] Debug logging is now hidden by default. See #13042
changelog_end

Summary of changes:
- remove unused `exports_files`
- refactor trigger-service-binary rules in a single comprehension
- add the `logback.xml` file in the JAR root

Tested manually: listed the contents of the output of `bazel build //triggers/service:trigger-service-binary-ce_deploy.jar` and `bazel build //triggers/service:trigger-service-binary-ee_deploy.jar` before and after the fix, observed that the only difference is the `logback.xml` file in the JAR root; added a debug logger call and ran both JARs before and after the fix, observed that after the fix the debug logging line is no longer printed.
2022-03-15 16:03:01 +00:00
Stephen Compall
153a5e6bcc
parallelize long-running custom/user token tests (#13290)
Saves about 90 seconds in test wallclock time. Other tests considered
either ran too short to bother splitting, or were Oracle and thus won't
affect the main run at all.

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-15 10:59:36 -04:00
Gary Verhaegen
dfa648f585
hunt down DAML better (#13195)
Process:

- `git ls-files -z | xargs -0 -n 100 sed -i --follow-symlinks 's/DAML/Daml/g'`
- `git add -p`
- `git restore -p`
- Check there is no unstaged change left.

To review:

- Check for false positives by carefully reviewing the diff in this PR.
- Check for false negatives with `git grep DAML`.
- Quicker check for fals positives:

```
git grep DAML | grep -v migration | grep -v DAML_
```

Fixes #13190

Note: This is the "second half" of #13191, which failed to cover all the
remaining DAMLs because of:

```
$ git ls-files | grep "'"
compiler/damlc/tests/daml-test-files/MangledScenario'.daml
```

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-08 17:04:58 +01:00
Moisés Ackerman
2dbd91aab9
Interface methods can use plain function declaration syntax (#13102)
* Update 'implements' blocks to use new syntax

* Add InterfaceSyntax test case

* Add InterfaceMultipleMethodDeclsError test case

* Add InterfaceDifferentNumArgsError test case

changelog_begin
changelog_end
2022-03-02 15:59:01 +00:00
Stephen Compall
7c1cf567ad
use user token for trigger service sandbox test fixture (#13077)
In SandboxFixture when mixing in the auth middleware fixture, set
up the ledger client with a user token instead of a claims token
when not running in the claims-token-specific auth tests.

Fixes #12831.

* note about inClaims

* prevent Product, Serializable warts with AuthServiceJWTPayload

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-25 14:08:44 -05:00
Stephen Compall
786906f23b
enable oauth2-test's user tokens in trigger service tests (#12994)
Duplicates the trigger service's auth tests for the two modes of
the oauth test server from #12929, then selects out the
claim-token-dependent tests as explained in
https://github.com/digital-asset/daml/issues/12831#issuecomment-1048176312

Part of #12831.

* also suppress claims-only tests in oracle-with-oauth2-test

* use inClaims for several tests

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-24 12:41:47 -05:00
Remy
e834a7b85c
Triggers: Connect Logging context from Triggers to the Speedy. (#13009)
Follow up of #12976.
fixes #12208

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-22 18:28:49 +01:00
Stephen Compall
6bb438e855
test middleware user tokens with oauth2 server user tokens (#12991)
TestMiddlewareUserToken now uses user tokens from the oauth
server as well, courtesy #12929. TestMiddlewareClaimsToken is the
only middleware-only (non-Client) test that switches the oauth
test server back to producing claims tokens, and contains all the
tests that depend on claims token semantics. These tests are in a
sense exercising the oauth server rather than the middleware.

The token returned by the oauth server is irrelevant for the behavior these
tests are exercising, so they are merely ported to always run on user tokens.

- CallbackUriOverride
- LimitedCallbackStore
- ClientLimitedCallbackStore
- ClientNoRedirectToLogin
- ClientYesRedirectToLogin
- ClientAutoRedirectToLogin

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-22 11:08:15 -05:00
Remy
ac3d9a4647
LF: Add LoggingContext to Speedy Machine (#12976)
This advances the state of #12208

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-21 13:34:46 +01:00
Stephen Compall
3ae19d991d
port auth middleware "test client" for user tokens (#12929)
Fixes #12881 by making the oauth2 test server invent user tokens
instead of claim tokens if you ask it to. As with the claim
tokens, these aren't "real" tokens backed by a participant
server, so only a subset of the tests makes sense, and we
abstract over the shared compatible tests accordingly.

CHANGELOG_BEGIN
CHANGELOG_END

* leave trigger service port for already-submitted issue #12831

* spin off another test idea, #12989
2022-02-17 19:44:08 +00:00
Stephen Compall
8ad957530f
port oauth2.TestMiddleware to test user tokens as well (#12855)
CHANGELOG_BEGIN
CHANGELOG_END
2022-02-10 16:16:31 +00:00
tudor-da
14a0459f0a
SandboxNextFixture replaced by SOX-based SandboxFixture (#12654)
* SandboxNextFixture replaced by Sandbox-on-X based SandboxFixture

changelog_begin
changelog_end

* Some fixed tests

* No direct dependencies on //ledger/sandbox:sandbox and //ledger/sandbox:sandbox-scala-tests-lib

* Fix after rebase

* Rename SandboxFixture and add a missing dep

* Generate valid party names if hint is empty

* Smaller maxInboundMessageSize

* Added test for empty display name

* SandboxServer is a ResourceOwner
* Uses execution context passed as an input for resource management
* Fixes flaky FlywayMigrations issue with null Thread.currentThread.currentClassLoader
* SandboxServer simplification returns Port instead of ApiServer

* Dedicated PMAllocateWithoutDisplayName for non-Canton ledgers
* Created since Canton does not return empty display names
2022-02-07 09:25:22 +01:00
Stephen Compall
6bdb901127
user tokens in oauth2-middleware (#12563)
* operate tokenProvidesClaims on user management service output

* ignore everything but the applicationId claim for StandardJWTPayload

* let the client tester work with non-custom tokens

* test claims check on non-custom token

* add changelog

CHANGELOG_BEGIN
- [Auth Middleware] Supports standard auth tokens for participant user management.
  See `issue #12563 <https://github.com/digital-asset/daml/pull/12563>`__.
CHANGELOG_END
2022-02-04 11:29:10 -05:00
Brian Healey
e70237a673
upgrade scalapb/netty/grpc/protobuf in proven combination (#12628)
* upgrade scalapb/netty/grpc/protobuf in proven combination

CHANGELOG_BEGIN
Upgrade scalapb, netty, grpc, protobuf and guava versions
CHANGELOG_END

* bazel reformat

* match grpc version in deps.bzl

* upgrade akka

* keep grpc version to 1.43 that is used in latest nixpkgs-unstable

* rebase and regen

* update SHA for scalapb tarball

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2022-02-03 08:38:38 -05:00
Remy
5cdf9b95fb
Remove some unecessary implictly with Factory (#12559)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-26 13:28:26 +00:00
Moritz Kiefer
16a4f0626b
Avoid explicitly listing all Oracle tests in build.yml (#12594)
changelog_begin
changelog_end
2022-01-26 11:28:44 +00:00
Marton Nagy
54339ada82
Safeguard Oracle CI tests with lockIdSeed [DPP-802] (#12573)
* Fixes OracleAround so it creates unique oracle users
* Fixes rouge connection pool in JdbcLedgerDaoTransactionsSpec
* Fixes cleanup in OracleAroundAll
* Introduces lockIdSeed for test frameworks
* Adapts usage

changelog_begin
changelog_end
2022-01-26 00:54:17 +01:00
Stefano Baghino
f1cd4b1c7c
Remove dependencies on compatibility libraries (#12548)
Continues the work started in https://github.com/digital-asset/daml/pull/12543

These libraries were only needed to transition from Scala 2.12 to 2.13
and are no longer useful as all the necessary items are now available
in Scala 2.13.

changelog_begin
changelog_end
2022-01-24 18:04:07 +00:00
Stefano Baghino
aec3390904
Replace silencer plugin with built-in warning configuration (#12543)
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.

[Here][1] is s a blog post by the Scala team about it.

Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.

Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.

changelog_begin
changelog_end

[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
2022-01-24 15:01:35 +00:00
Moritz Kiefer
688f1e1e0b
Drop v0 contract ids (#12464)
* Drop v0 contract ids

changelog_begin
changelog_end

* .

changelog_begin
changelog_end
2022-01-20 15:28:27 +00:00
Moritz Kiefer
b11f11bde0
Drop support for no seeding in sandbox-classic (#12495)
* Drop support for no seeding in sandbox-classic

Sandbox classic is going away in SDK 2.0 and so is support for v0
contract ids which is the only thing no seeding was used for.

This has been cleared by product.

I’ll drop v0 contract ids completely in #12464 as a follow-up. Just
wanted to factor this out for ease of review.

changelog_begin
changelog_end

* .

* .
2022-01-20 09:15:13 +01:00
Stefano Baghino
4af48bbccc
Split channel configuration from LedgerClientConfiguration (#12433)
* Split channel configuration from LedgerClientConfiguration

Fixes #12391

The channel configuration now has to be provided separately from the
configuration specific to the ledger client. In this way we avoid
situations where the builder is provided with some configuration
that gets overridden.

changelog_begin
[Scala bindings] The channel configuration has been split from the
LedgerClientConfiguration class. Provide the gRPC channel specific
configuration separately or use a builder. The channel configuration
no longer overrides the builder.
changelog_end

* Fix compilation issues in //ledger-service/...
2022-01-18 10:03:33 +00:00