Commit Graph

48 Commits

Author SHA1 Message Date
Stefano Baghino
aec3390904
Replace silencer plugin with built-in warning configuration (#12543)
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.

[Here][1] is s a blog post by the Scala team about it.

Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.

Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.

changelog_begin
changelog_end

[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
2022-01-24 15:01:35 +00:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Samir Talwar
9f8e640b1a
Use ExecutionContext.parasitic instead of DirectExecutionContext. (#11783)
* concurrent: Replace `DirectExecutionContextInternal` with `parasitic`.

* concurrent: Rename `DirectExecutionContext` `parasitic`.

* Use `ExecutionContext.parasitic` instead of `DirectExecutionContext`.

We no longer need the latter.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix formatting.
2021-12-10 16:25:46 +01:00
Marton Nagy
308f938512
Dpp 494 unit testing ha coordinator (#10862)
* Unit testing: HA Coordinator

* Preparation: switch to fix-thread-pool in AkkaBeforeAndAfterAll to have more stable test runtimes

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: switch to timer from akka-pattern  and Scheduler in waiting-futures, for higher precision timing

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation: prevent race in PollingChecker (corner-case uncovered via sustained testing)

CHANGELOG_BEGIN
CHANGELOG_END

* Preparation - HaCoordinator refactorings: switch from DataSource to connection factory functions to clean dependencies, fixing some typos

CHANGELOG_BEGIN
CHANGELOG_END

* Add test infrastructure for locking

CHANGELOG_BEGIN
CHANGELOG_END

* Reuse StorageBackendTestsDBLock to test TestDBLockStorageBackend

CHANGELOG_BEGIN
CHANGELOG_END

* Add unit test suite for HaCoordinator

CHANGELOG_BEGIN
CHANGELOG_END

* Fix random generation to respect scala 2.12

CHANGELOG_BEGIN
CHANGELOG_END

* Adds test cases for graceful-shutdown during initialization

CHANGELOG_BEGIN
CHANGELOG_END

* Minor changes based on review

CHANGELOG_BEGIN
CHANGELOG_END

* Some comment rewording based on review

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-16 17:08:45 +02:00
Samir Talwar
4b8b67a1b5
Upgrade Scalatest to v3.2.9. (#10576)
* Upgrade Scalatest to v3.2.9.

Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.

The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.

As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.

CHANGELOG_BEGIN
CHANGELOG_END

* http-json-oracle: Fix a Scalatest dependency.

* ledger-api-client: Fix a Scalatest dependency.
2021-08-12 23:19:35 +00:00
Miklos
0eba812109
Remove trace_context field from Ledger API [KVL-1021] (#10256)
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
2021-07-15 19:06:25 +02:00
Stephen Compall
ca9e89b3da
check whether collection.compat is unused when compiling for Scala 2.12 (#9604)
* check whether collection.compat is unused when compiling for Scala 2.12

- Instead of always suppressing warnings for collection.compat._,
  we should only do it for Scala 2.13

- We can also reduce boilerplate by automatically adding this
  option when both silencer_plugin and collection-compat are
  present

CHANGELOG_BEGIN
CHANGELOG_END

* remove unused import

* remove another unused import

* remove even more unused imports

* missed compat dependency

* more missed compat dependencies

* missed compat dependency

* use scala_deps in scaladoc_jar

- #8423 inlined the major version expansion, but this seems to
  have been prior to proper support by scaladoc_jar

* restore custom handling of participant-integration-api

- fixing scaladoc_jar isn't worth it for a single case, as with
  deps vs scala_deps
2021-05-11 21:54:14 +00:00
Samir Talwar
e1af564bcc
Switch from @silent to @nowarn. (#9498)
* Switch from `@silent` to `@nowarn`.

This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.

I had to get creative about a couple of use cases that didn't work.
Specifically:

  1.  Suppressing deprecation warnings works, but Scala 2.12 erroneously
      complains that the `@nowarn` is unnecessary.  I had to suppress
      this warning too with `-Ywarn-unused:-nowarn`.
  2.  I can't seem to suppress the warning, "The outer reference in this
      type test cannot be checked at run time." Instead, I have
      refactored the code to remove the warning.

We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.

CHANGELOG_BEGIN
CHANGELOG_END

* Add some comments around `@nowarn` support.

* language-support/scala: Fix a warning suppression.

* Revert to the default warnings.

Compatibility was complaining.
2021-04-26 19:46:14 +00:00
Hubert Slojewski
51f495e758
KVL-203 Share participant state integration test harness (#9143)
* Expose libraries for integration testing purposes

The motivation of these changes is to eliminate manual work and reduce duplication between the SDK and oem-integration-kit repos by reusing the same test fixture for integration testing participant state implementations. Also, the DARs required for running these tests won't need to be manually updated.

CHANGELOG_BEGIN
CHANGELOG_END

* Fix a concurrency issue in integration tests

* Fix Bazel error

* Fix conflict resolution

* Move inline daml-lf to separate dar files

* Add a comment

* Add a missing artifact

* Extract method

* Remove maven tags

* Add a macro for Scala libraries with dar resources

* Improve the macro

* Add missing artifact

* Simplify the tests

* Format signature

* Fix the maven tag

* Add missing copyright headers

* Format bazel files

* Make //ledger/test-common lf version dependent (to avoid jar hell)

* Move da_scala_dar_resources_library to a separate bzl file

* Add missing artifacts

Co-authored-by: Hubert Slojewski <hubert.slojewski@tesco.com>
2021-03-19 16:29:02 +01:00
Moritz Kiefer
22ce940954
Upgrade scalafmt and enable trailing commas (#8437)
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.

As announced, this will be merged on Saturday to avoid too many conflicts.

changelog_begin
changelog_end
2021-01-09 11:37:37 +01:00
Gary Verhaegen
a925f0174c
update copyright notices for 2021 (#8257)
* update copyright notices for 2021

To be merged on 2021-01-01.

CHANGELOG_BEGIN
CHANGELOG_END

* patch-bazel-windows & da-ghc-lib
2021-01-01 19:49:51 +01:00
Moritz Kiefer
1b32044a74
Build //libs-scala/... on 2.13 (#8282)
* Build //libs-scala/... on 2.13

One test is unfortunately disabled at the moment since I utterly
failed to figure out why I get a ClassNotFoundException on 2.13.

changelog_begin
changelog_end

* Copyright headers

changelog_begin
changelog_end

* I can’t bazel today

changelog_begin
changelog_end

* Apply suggestions from code review

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>

* Update libs-scala/resources/src/main/2.13/com/daml/resources/UnitCanBuildFrom.scala

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

* No split on view

changelog_begin
changelog_end

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-12-14 17:36:53 +00:00
Moritz Kiefer
7e05dc7932
Upgrade rules-scala and scalatest (#8187)
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.

This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.

Apologies for the giant PR, I don’t see a way to keep it smaller.

changelog_begin
changelog_end
2020-12-08 06:59:23 +01:00
Stefano Baghino
bc0f6ab35e
[DPP-84] Employ parallel unnesting instead of batching on PostgreSQL (#8042)
* [DPP-84] Employ parallel unnesting instead of batching on PostgreSQL

changelog_begin
[Integration Kit] When using a PostgreSQL-based index, leveraging native
parallel unnesting allows to more efficiently index new transactions.
changelog_end

* Address review https://github.com/digital-asset/daml/pull/8042#pullrequestreview-541759596
2020-12-01 12:06:36 +00:00
Samir Talwar
052f69cde9
ledger-api: Use proto_jars, and publish Protobuf sources separately from the Scala classes. [KVL-714] (#8091)
* ledger-api: Use `proto_jars`.

CHANGELOG_BEGIN
- [Ledger API] The Scala JARs containing the gRPC definitions no longer
  contain the *.proto files used to generate the ScalaPB-based classes.
CHANGELOG_END

* Create a source JAR for *.proto files in `proto_jars`.

* ledger-api: Publish the protobuf sources as "ledger-api-proto".

CHANGELOG_BEGIN
- [Ledger API] The *.proto files containing the gRPC definitions are now
  provided by a new Maven Central artifact, with the group "com.daml"
  and the artifact name "ledger-api-proto".
CHANGELOG_END

* release: We don't need the "main-jar" option.

* Bazel: Proto JARs will always have a Maven artifact suffix.

* Bazel: Simplify Protobuf source file TAR and JAR targets.

* Bazel: Extract out Protobuf functions.
2020-11-27 17:14:48 +00:00
Samir Talwar
7f679b9896
resources: Customizable contexts. (#7678)
* resources: Move builders into //ledger/ledger-resources.

Keep the actual constructors in a trait, but instantiate it when working
with ledger code.

This allows us to later introduce an extra "context" type parameter to
ResourceOwner.

* resources-akka: Move the builders in to //ledger/ledger-resources.

* resources: Introduce an abstract `Context` parameter for owners.

This replaces the concrete `ExecutionContext`. While it _can_ be an
execution context, it really doesn't matter as long as we can get at one
somehow.

This is being introduced so we can wrap the context in a container,
either for type tagging or to include extra information.

Because our current context _is_ `ExecutionContext`, and an implicit is
provided to extract it, we can end up with two ways to get the same
value. We use shadowing to prevent this. This problem should go away in
the near future when a new context type is added.

CHANGELOG_BEGIN
- [Integration Kit] The `ResourceOwner` type is now parameterized by a
  `Context`, which is filled in by the corresponding `Context` class in
  the _ledger-resources_ dependency. This allows us to pass extra
  information through resource acquisition.
CHANGELOG_END

* ledger-resources: Move `ResourceOwner` here from `resources`.

* ledger-resources: Remove dependencies from outside //ledger.

* ledger-resource: Wrap the acquisition execution context in `Context`.

So we can add a logging context to it.

* resources: Pass the Context, not the ExecutionContext, to Resource.

* Avoid importing `HasExecutionContext`.

* ledger-resources: Publish to Maven Central.

* resources: Make the small changes suggested by @stefanobaghino-da.

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* ledger-resources: Pull out a trait for test resource contexts.

Saves a few lines of code.

* Restore some imports that were accidentally wildcarded.

* resources: Replace an `implicit def` with a couple of imports.

* participant-integration-api: Simplify the JdbcLedgerDaoBackend tests.

Try and use the right execution context where possible.

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-10-20 09:26:28 +00:00
Samir Talwar
f0c1eb207c
concurrent: Tag DirectExecutionContext. (#7517)
* concurrent: Tag DirectExecutionContext.

1.  Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
    stating that it works for any tagged `Future`.
2.  Move `DirectExecutionContext` to the _libs-scala/concurrent_
    library, as it requires it and it's tiny.

CHANGELOG_BEGIN
CHANGELOG_END

* concurrent: Fix the privacy of `DirectExecutionContextInternal`.

Co-authored-by: Stephen Compall <stephen.compall@daml.com>

Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2020-09-29 17:23:57 +00:00
Stefano Baghino
e972872128
Properly use LoggingContext in Participant Server (#6924)
* Properly use LoggingContext in Participant Server

Fixes #6837

Spreads usage of LoggingContext and ContextualizedLogger throughout the participant server.

changelog_begin
[Sandbox/Integration Kit] We have enriched the contextual information
exposed by the Ledger API server. You should note richer logging information,
which can be read either via unstructured or structured logging frameworks.
A paragraph on how to configure structured logging has been added to the docs.
For more on the issue, see https://github.com/digital-asset/daml/issues/6837.
changelog_end

* Make everything compile

* Leave only actual commands on trace level on submission

* Shorter log messages on submission

* Don't add values down the call stack

* Document good practices regarding the logging context

* Add context to write service implementations

* Add logging context to tests

* Document logging for DAML on SQL

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463503013

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463508117

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463515665

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463513157

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463516359

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463516895

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463518813

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463520210

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463521501

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463521593

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463525453

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463525560

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463525672

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463525742

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463526837

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527054

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527523

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527814

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527958

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527900

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463527997

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463528050

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463542877

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463543051

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463543614

* Address https://github.com/digital-asset/daml/pull/6924#discussion_r463543692
2020-07-31 12:40:13 +00:00
Gerolf Seitz
a2d785e3ee
Use com.daml as root package (#5343)
Packages com.digitalasset.daml and com.daml have been unified under com.daml

Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.

CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
2020-04-05 19:49:57 +02:00
Samir Talwar
33b55465cb
testing-utils: Make ManagedResource idempotent. (#5388)
We saw a few issues in CI where `setup` might be called twice, which
resulted in a test fail for no good reason. Rather than crashing, we
can just do nothing the second time.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-03 07:19:27 +00:00
Samir Talwar
1bedfaa151
testing-utils: Actually run the tests. (#5381)
There aren't many, but they deserve to be run.

CHANGELOG_BEGIN
CHANGELOG_END
2020-04-02 18:56:39 +00:00
Gerolf Seitz
329320bad9
Organize maven coordinates (#5272)
* Use com.daml as groupId for all artifacts

CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END

* Add 2 additional maven related checks to the release binary

1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId

* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
2020-04-01 11:41:18 +02:00
Gary Verhaegen
1872c668a5
replace DAML Authors with DA in copyright headers (#5228)
Change requested by Manoj.

CHANGELOG_BEGIN
CHANGELOG_END
2020-03-27 01:26:10 +01:00
Robert Autenrieth
a3ddde3a9d
Remove MRT and LET (#5172)
Contributes to #4194.
Closes #4231.
Closes #5022.
CHANGELOG_BEGIN
- [Ledger API] The protobuf fields ledger_effective_time and maximum_record_time have been removed from
  command submission. These fields were previously deprecated following the introduction
  of a new ledger time model. See issue `#4194 <https://github.com/digital-asset/daml/issues/4194>`__.
[Java Bindings] removed the usage of ledgerEffectiveTime and
maximumRecordTime, and instead added minLedgerTimeAbsolute and
minLedgerTimeRelative in CommandSubmissionClient and CommandClient
CHANGELOG_END
2020-03-25 16:50:27 +01:00
Samir Talwar
a589f4af0b
sandbox: Move more resource acquisition into the owner. (#4501)
* sandbox: Move more resource acquisition into the `owner`.

CHANGELOG_BEGIN
CHANGELOG_END

* sandbox: Reimplement SandboxClientResource as a resources.Resource.

* codegen: Use resources in TestUtil.

* sandbox: Manage PostgreSQL in tests with ResourceOwners.
2020-02-14 13:52:45 +00:00
Gerolf Seitz
85db1033d7
Fix memory leak when using the reset service (#4447)
* sandbox: Don't hold on to old resources when resetting.

Now there's one hell of a memory leak.

CHANGELOG_BEGIN
- [Sandbox] Fixed a memory leak when using the ResetService; not
  everything was cleaned up correctly.
CHANGELOG_END

* sandbox: Split out SandboxClientResource from SandboxServerResource.

Gonna replace SandboxServerResource with a ResourceOwner acquisition.

* sandbox: Don't capture the API server in the SandboxServer resource.

When we reset, this is stored forever, leading to a memory leak.

Tested by rewriting the SandboxServerResource to use
`SandboxServer.owner`.

* sandbox: Revert the test client resource to calling `shutdownNow()`.

* sandbox: Make sure the fixture is recreated properly on each test run.

* sandbox: Make `SandboxState` a non-case class.

The `toString()` was unnecessarily heavy.

* sandbox: Futures, futures everywhere.

Avoid a race condition where the server is stopped before it starts by
storing a `Future[SandboxState]` rather than the `SandboxState` itself.

This doesn't trigger the same memory leak as storing a
`Resource[SandboxState]` because we don't capture the object itself in
the `flatMap` in the same way with `Future`.

* sandbox: Remove an unused parameter left in for debugging.

* sandbox: Replace `@VisibleForTesting` with a comment.

* sandbox: Add more comments to the weird logic in SandboxServer.

* sandbox: Get rid of the `Port` type alias; it was confusing.

Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
2020-02-10 16:47:39 +00:00
Gary Verhaegen
878429e3bf
update copyright notices to 2020 (#3939)
copyright update 2020

* update template
* run script: `dade-copyright-headers update .`
* update script
* manual adjustments
* exclude frozen proto files from further header checks (by adding NO_AUTO_COPYRIGHT files)
2020-01-02 21:21:13 +01:00
Stefano Baghino
7020ed3b15
Moves CommandCompletionIT to Ledger API Test Tool (#3901)
* Moves CommandCompletionIT to Ledger API Test Tool

Closes #2841

Also refactors test observers into a single API and unifies the existing implementations of the DirectExecutionContext

* Allow explicit offsets to get checkpoints

* Raise checkpoint test timeout

* Consolidate checkpoint tests in a single test case

* Fix compilation issues

* Exclude command submission/completion tests for rev-v2

* Drop test not relevant for Ledger API implementations

* Fix compilation errors
2019-12-20 15:55:23 +01:00
Brian Healey
9f13a2fb10 Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10 (#3903)
* Upgrade to Akka 2.6.1, akka-http 10.1.11 and Scala 2.12.10

Akka 2.6.1 Upgrade Changes
- Materializer in place of ActorMaterializer
- Source.future instead of Source.fromFuture
- The Scheduler.schedule method has been deprecated in favor of selecting scheduleWithFixedDelay or scheduleAtFixedRate
- onDownstreamFinish(cause: Throwable)
- ActorAttributes.supervisionStrategy(...) in place of ActorMaterializerSettings.withSupervisionStrategy

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* Akka 2.6.1 Upgrade Changes
- onDownstreamFinish(cause: Throwable)

See https://doc.akka.io/docs/akka/current/project/migration-guide-2.5.x-2.6.x.html

* code review: remove unnecessary supervision strategy
2019-12-19 18:00:59 +00:00
Stefano Baghino
3ca7ed49dd Remove warts: ExplicitImplicitTypes (#3851)
* Remove warts: ExplicitImplicitTypes

* Fix compilation errors
2019-12-13 18:33:49 +00:00
Samir Talwar
0ff716df2a Ledger API: Add healthcheck endpoints. (#3573)
* grpc-definitions: Delete health_service.proto

We can use the version in io.grpc:grpc-services instead.

* ledger: Delete ledger/API.md.

* sandbox: Fix warnings in ApiServices flagged by IntelliJ.

* sandbox: Implement a dummy grpc.health.v1.Health.Check endpoint.

* sandbox: Implement a dummy grpc.health.v1.Health.Watch endpoint.

* sandbox: Drop repeated elements from grpc.health.v1.Health.Watch.

* sandbox: Wrap the HealthService in basic tests.

* sandbox: Stop streaming the server health too.

* ledger-api-test-tool: Health check tests.

* Add a changelog entry for the health check endpoints.

CHANGELOG_BEGIN

- [Ledger API] Add healthcheck endpoints, conforming to the
  `GRPC Health Checking Protocol <https://github.com/grpc/grpc/blob/master/doc/health-checking.md>`_.
  It is always ``SERVING`` for now.

- [DAML Ledger Integration Kit] Add conformance test coverage for the
  ``grpc.health.v1.Health`` service.

CHANGELOG_END

* ledger-api-integration-tests: Increment the number of services.

* Apply suggestions from code review

Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* sandbox: Use `AkkaBeforeAndAfterAll` in the HealthServiceSpec.

In an attempt to get it working on CI.

* sandbox: Change `dropRepeated` to `DropRepeated()`.

Keep it in one file.

* test-common: Use `Delayed.by` in `TimeBoundObserver`.

* test-common: Close the source when `TimeBoundObserver` completes.

* ./fmt.sh

That'll teach me not to `--no-verify` just because it's a merge commit.

* sandbox: Inline `HealthService.suppress`.

At some point it was being used twice.

* sandbox: Increase the timeout for HealthServiceSpec.

* sandbox: Reimplement HealthService using the Scala protobuf types.

* sandbox: Generate an Akka-compatible trait for the health service.

And refactor a lot of test code to make it easy to test.

* ledger-api-common: Move the HealthService here.

* rs-grpc-testing-utils: Publish to Maven.

* rs-grpc-testing-utils: Add Maven coordinates.
2019-11-22 14:02:05 +00:00
Stefano Baghino
c51ecd3c58
Extractor with authentication (#3514)
* Make Extractor work against Ledger API servers protected by authentication

* Update changelog

* Address https://github.com/digital-asset/daml/pull/3514#pullrequestreview-318811500

- https://github.com/digital-asset/daml/pull/3514#discussion_r347753205
- https://github.com/digital-asset/daml/pull/3514#discussion_r347753629
- https://github.com/digital-asset/daml/pull/3514#discussion_r347754462

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347755484

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347752424

* Address https://github.com/digital-asset/daml/pull/3514#discussion_r347756328

* Fix docs

* Fix artifacts.yaml
2019-11-19 17:12:25 +01:00
Stefano Baghino
c8aaffd62c Add GrpcException extractor (#3344)
* Add GrpcException extractor

* Use GrpcStatus in integration test

* Fix compilation error

* Fix compilation error

* Address https://github.com/digital-asset/daml/pull/3344#discussion_r342981204

* Restricting extractor types
2019-11-06 14:44:06 +00:00
Stefano Baghino
edb2d15cec
Port command-related integration tests (#3320)
* Pre-port cleanup of helpers

* Further clean-up

* Remove dead code

* More cleaning and flattening

* Port first tests from CommandClientIT

* More cleanup and more dead code removal

* Port exercise by key test

* Port observer disclosure tests

* Port tests, remove duplicates and dead code

* Fix compilation error and remove more dead code

* Remove unused import

* Spread helpers to single users, remove materializer from helpers

* More test porting and cleanup

* Remove more dead code

* Restore tests deleted by mistake

* Restore final qualifier removed by mistake

* Fix typo

* Optimize imports

* Move timeout from helper to tests

* Remove unused class

* Never gonna give you up

* Address https://github.com/digital-asset/daml/pull/3320#discussion_r342018581

* Address https://github.com/digital-asset/daml/pull/3320#discussion_r342025161

* Remove removed test :)
2019-11-04 17:30:31 +01:00
Andreas Herrmann
2bd1db490a
Replace bazel-deps by rules_jvm_external (#3253)
* Update bazel-common to fix javadoc issues

Specifically, to fix the following error

```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel:7:1: in javadoc_library rule //ledger-api/rs-grpc-bridge:rs-grpc-bridge_javadoc:
Traceback (most recent call last):
        File "/home/aj/tweag.io/da/da-bazel-1.1/ledger-api/rs-grpc-bridge/BUILD.bazel", line 7
                javadoc_library(name = 'rs-grpc-bridge_javadoc')
        File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/com_github_google_bazel_common/tools/javadoc/javadoc.bzl", line 27, in _javadoc_library
                dep.java.transitive_deps
object of type 'JavaSkylarkApiProvider' has no field 'transitive_deps'
```

* Define Maven deps using rules_jvm_external

* Pin artifacts

* Remove bazel-deps generated targets

* Remove bazel-deps

* Switch to rules_jvm_external targets

* update bazel documentation

* pom_file: There are no more bazel-deps targets

* BAZEL-JVM.md `maven_install` typo
2019-10-28 13:53:14 +01:00
Gary Verhaegen
99ea93168d
update copyright notices (#2499) 2019-08-13 17:23:03 +01:00
Stefano Baghino
afa4659292
New Ledger API test tool (#2282)
* New acceptance test tool prototype

* Fix transaction filter helper

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308081386

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308082432

Moreover, creates specific helpers for test DAML code.

* Move helpers from session to context

Remove a circular dependency between session and context, and limits the
usage of implicits.

* Run tests with ledgers out of process

* Prepare for drop in

* Temporary timeout increase

* Move bulk of the logic to the test runner

* Have a map of tests for CLI integration

* Timeout per-test, runner uses context, renaming

* Allow usage of time service, integrate semantic tests

* Integrate CLI SSL configuration

* Integrate failure expectation CLI option

* Integrate with failure output verbosity CLI option

* Integrate with test listing and picking CLI options

* Integrate with timeout scaling CLI option

* Integrate with command TTL CLI option

* Review CODEOWNERS

* Drop previously unimplemented CLI parameters

* Expand test todo list to encompass latest additions

* Deduplicate some testing infrastructure, swap in new test tool

* Remove unused rule

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311045444

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311051974

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311035287

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311055290

* WIP

* Make gRPC thread factory spawn non-deamon threads

* Remove debug prints, complete divulgence test (missing assertion messages)

* Try to detect assertion failure line number

* Add Sandbox binary that starts an ephemeral postgres instance

This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.

Fixes #1543

* Add Sandbox binary that starts an ephemeral postgres instance

This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.

Fixes #1543

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311400695

* Remove TlsConfiguration duplicate, move original to commons

* Make assertion more readable

Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>

* Fix build errors introduced in previous commit

- exclude time service tests from reference-v2
- add copyright header to TlsConfiguration

* Complete divulgence test and address review comments

Restores comments from original divulgence tests and strives to be as
close to them as possible.

- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311429169
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431156
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431913

* Delete old divulgence tests

* Restore methods to fetch contracts by template identifier

* Run conformance tests on in-memory sandbox

* Run conformance tests on the persistent sandbox

* Remove workflow identifier workaround

* New contract keys test (#2452)

* Set channel type for LedgerSession

* Add ContractKeysIT

* Run tests against reference-server v2

* Removed the old ContractKeysIT. However, ...

... I cannot remove ContractKeysChecks completely, because we have
another test path for DARs created prior to DAML-LF 1.6, which doesn't
have the submitter == maintainer restriction for contract keys.

* Use codegen instead of custom code

* Add missing copyright headers

* Drop sandbox conformance test suite (seems to not work on Windows CI)

* Make time simpler

* Test tool tests to run only semantic tests

* Make CI behave more consistently

* Add missing files to run tests on Postrgres

* Fix flaky divulgence test, introduce readable identifiers

* Increase sandbox conformance tests timeout

* Address https://github.com/digital-asset/daml/pull/2282#discussion_r312363246

* Reintroduce test resource extraction

* Read semantic tests from the classpath rather than Bazel

* Use codegen types for ContractKeys integration test

allocateParty returns Primitive.Party and create and exercise also
accept Primitive.Party. This way we don't have to keep a String and
Primitive.Party representation around.

* Bump timeout for sandbox conformance tests

* Remove dependency on Bazel runfiles
2019-08-09 17:00:37 +02:00
Gerolf Seitz
136f6d1c08
Remove ExercisedEvent#contract_creating_event_id. (#2262)
This is a breaking change on the ledger api. So far we could rely on the
assumption that contractId == eventId. This assumption doesn't hold
anymore in a daml-on-x setting, where the eventId is created by the
api server, but the absolute contractId is created by the ledger
implementation.

Instead of going through weird contortions to store the relevant data in
the existing database schema, it is more viable in the long term to
remove that field and instead provide facilities to lookup transactions
by contractId.

Contributes to #2068.
2019-07-29 11:30:01 +02:00
Remy
53de5edbc1 ledger-api: Kill legacy identifier (#2211)
* ledger-api: drop support for legacy identifier

* address scala-codegen: Stephen's comments

* language support: fix test
2019-07-23 20:34:52 +00:00
Stefano Baghino
4774e75eb8 The completion stream RPC defaults to the ledger end as offset (#1961)
* The completion stream RPC defaults to the ledger end as offset

Fixes #1913

Relevant changes are propagated to the Java bindings (including
deprecating a method that would now return a nullable ledger end).

* Refactor completionStream method

* Address review comments

- ignore command creation results
(https://github.com/digital-asset/daml/pull/1961#discussion_r299089539)
- avoid re-connecting to the client for every command
(https://github.com/digital-asset/daml/pull/1961#discussion_r299092328)
- move offset field optionality to domain object
(https://github.com/digital-asset/daml/pull/1961#discussion_r299090451)

* Improve tests
2019-07-02 10:02:43 +00:00
Stefano Baghino
6ba26f4e1f Add contract keys to created events in the Ledger API (#1586)
* Add contract keys to created events in the Ledger API

Fixes #1268

* Update ledger/ledger-api-integration-tests/src/test/itsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/TransactionServiceIT.scala

Co-Authored-By: Stephen Compall <scompall@nocandysw.com>

* Update language-support/java/bindings-rxjava/src/test/scala/com/daml/ledger/rxjava/grpc/helpers/TransactionGenerator.scala

Co-Authored-By: Stephen Compall <scompall@nocandysw.com>

* Update ledger/ledger-api-integration-tests/src/test/itsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/TransactionServiceIT.scala

Co-Authored-By: Stephen Compall <scompall@nocandysw.com>

* Add run suffix to transaction service integration tests

Addresses https://github.com/digital-asset/daml/pull/1586#discussion_r293233810
Addresses https://github.com/digital-asset/daml/pull/1586#discussion_r293233886

* Re-introduce unused bindings

Addresses https://github.com/digital-asset/daml/pull/1586#discussion_r293056685
Addresses https://github.com/digital-asset/daml/pull/1586#discussion_r293056741

* Update ledger/ledger-api-common/src/main/scala/com/digitalasset/platform/participant/util/LfEngineToApi.scala

Co-Authored-By: Stephen Compall <scompall@nocandysw.com>

* Use newly introduce assertOrRuntimeEx method

Applies the suggestion introduced by 7cc5c3eb71

* Contract keys to be typed after generated types

Also adds support in daml-lf/interface for contract keys

Addresses https://github.com/digital-asset/daml/pull/1586#pullrequestreview-249431707

* Address https://github.com/digital-asset/daml/pull/1586#discussion_r293773427

* Fix compilation error from previous commit

* Fix compilation errors in navigator
2019-06-14 16:45:12 +00:00
gleber
b61203d1cd Run Transaction Service IT as part of Ledger API Test Tool (#1434)
Add tTansactionServiceIT into Ledger API Test Tool as an optional test.

Available tests can be listed with the --list option.
Tests can be included with --include, and excluded with --exclude.

Fixes #1372, #1472
2019-06-06 13:32:55 +02:00
gleber
a69b4b8c86 Ledger API Tests: use real ledger ID everywhere. (#1400)
This fetches and caches ledger ID inside LedgerContext.SingleChannelContext to
reduce number of calls into the Ledger API under test.
2019-06-03 12:28:36 +02:00
gleber
40ce2b9be8 Unify semantic test between sandbox IT and Ledger API Test Tool (#1171)
* Add RemoteApiProxy fixture type.

This is in preparation for using Sandbox IT suite as part of the Ledger API Test
Tool.

* ledger-api-test-tool: Drop reset functionality.

This is no longer necessary for the tool and it does not scale with the types of
tests in the suite.

* integration-tests: Fail if the server under fixture is stuck.

This makes sure that a server getting stuck will get detected by a test, instead
of ignoring it and potentially allowing the server to linger.

* integration-test: Make semantic testing runs independent.

It manges parties and command identifier to include a unique (random) suffix in
all ledger-commited identifiers. This allows the test to run against a Ledger
API without reseting it.

* ledger-api-test-tool: Unify test code using scenario runner with IT suite.

This reuses the scenario runner test code from the IT suite, instead of
reimplementing it. This should be a no-op (except for tests reports formatting).

* Review fixes.

* Ledger API Test Tool: Provide logback config.

This quites Ledger API Test Tool output.

* Make sure akka threads are terminated at end of test runs.

This makrs Akka threads to be daemons, hence forcing them to be closed at the
end of Ledger Api Test Tool.

* Use Ledger API Test Tool in tests of reference server.

* Add Apache commons-lang3.

* Ledger API Test Tool: Implement custom test reporter.

This addresses two needs:
- avoid using buggy scalatest test reporter;
- pretty-prints test results prettier.

* dade-copyright-headers: return success on successful reformatting.
2019-05-28 09:58:58 +00:00
gleber
aa70c7f64e
Enforce consistent formatting of BUILD files. (#412)
* Add buildifier targets.

The tool allows to check and format BUILD files in the repo.

To check if files are well formatted, run:

    bazel run //:buildifier

To fix badly-formatted files run:

    bazel run //:buildifier-fix

* Cleanup dade-copyright-headers formatting.

* Fix dade-copyright-headers on files with just the copyright.

* Run buildifier automatically on CI via 'fmt.sh'.

* Reformat all BUILD files with buildifier.

Excludes autogenerated Bazel files.
2019-04-12 13:10:16 +02:00
moritzkiefer-da
fa4067ad1b
Move POM file generation to Bazel rules (#374)
* Move POM file generation to Bazel rules
2019-04-11 11:24:52 +02:00
Gabor Aranyossy
094b4b6959
Sandbox Sql fixture (#247)
* SandboxSQL fixture + fixing various issues

WIP - playing around with Resources to support Sandbox Fixture with Postgres

Sandbox with Postgres fixture works - LedgerIdentityServiceGivenIT PASSES

adding guards for empty collections in PostgresLedgerDao

CommandCompletionServiceITi passed for both fixtures

fixing EventID formatting problem

another formatting fix

changed wrong autoCommit behaviour

CommandStaticTimeIT passed on both ledgers

rolling back transaction on duplicates

dealing with duplicate commands in postgres

fixing mismapped record_time

adding extra logging for fatal errors

disabling sql support as it's not finished yet

cleanup

removed unnecessary test classes

cleanup

formatting

* ADT was missing Product with Serializable

* fixed the compilation error
2019-04-10 17:04:47 +02:00
Digital Asset GmbH
05e691f558 open-sourcing daml 2019-04-04 09:33:38 +01:00