daml/ledger/sandbox/BUILD.bazel

347 lines
12 KiB
Python
Raw Normal View History

# Copyright (c) 2021 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
2019-04-04 11:33:38 +03:00
# SPDX-License-Identifier: Apache-2.0
load("//bazel_tools:scala.bzl", "da_scala_binary", "da_scala_library", "da_scala_test_suite")
load("//ledger/ledger-api-test-tool:conformance.bzl", "server_conformance_test")
load("@os_info//:os_info.bzl", "is_windows")
introduce new release process (#4513) Context ======= After multiple discussions about our current release schedule and process, we've come to the conclusion that we need to be able to make a distinction between technical snapshots and marketing releases. In other words, we need to be able to create a bundle for early adopters to test without making it an officially-supported version, and without necessarily implying everyone should go through the trouble of upgrading. The underlying goal is to have less frequent but more stable "official" releases. This PR is a proposal for a new release process designed under the following constraints: - Reuse as much as possible of the existing infrastructure, to minimize effort but also chances of disruptions. - Have the ability to create "snapshot"/"nightly"/... releases that are not meant for general public consumption, but can still be used by savvy users without jumping through too many extra hoops (ideally just swapping in a slightly-weirder version string). - Have the ability to promote an existing snapshot release to "official" release status, with as few changes as possible in-between, so we can be confident that the official release is what we tested as a prerelease. - Have as much of the release pipeline shared between the two types of releases, to avoid discovering non-transient problems while trying to promote a snapshot to an official release. - Triggerring a release should still be done through a PR, so we can keep the same approval process for SOC2 auditability. The gist of this proposal is to replace the current `VERSION` file with a `LATEST` file, which would have the following format: ``` ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7 ``` This file would be maintained with a script to reduce manual labor in producing the version string. Other than that, the process will be largely the same, with releases triggered by changes to this `LATEST` and the release notes files. Version numbers =============== Because one of the goals is to reduce the velocity of our published version numbers, we need a different version scheme for our snapshot releases. Fortunately, most version schemes have some support for that; unfortunately, the SDK sits at the intersection of three different version schemes that have made incompatible choices. Without going into too much detail: - Semantic versioning (which we chose as the version format for the SDK version number) allows for "prerelease" version numbers as well as "metadata"; an example of a complete version string would be `1.2.3-nightly.201+server12.43`. The "main" part of the version string always has to have 3 numbers separated by dots; the "prerelease" (after the `-` but before the `+`) and the "metadata" (after the `+`) parts are optional and, if present, must consist of one or more segments separated by dots, where a segment can be either a number or an alphanumeric string. In terms of ordering, metadata is irrelevant and any version with a prerelease string is before the corresponding "main" version string alone. Amongst prereleases, segments are compared in order with purely numeric ones compared as numbers and mixed ones compared lexicographically. So 1.2.3 is more recent than 1.2.3-1, which is itself less recent than 1.2.3-2. - Maven version strings are any number of segments separated by a `.`, a `-`, or a transition between a number and a letter. Version strings are compared element-wise, with numeric segments being compared as numbers. Alphabetic segments are treated specially if they happen to be one of a handful of magic words (such as "alpha", "beta" or "snapshot" for example) which count as "qualifiers"; a version string with a qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`, which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a special ordering amongst qualifiers. Other alphabetic segments are compared alphabetically and count as being "after" their prefix (`1.2.3-really-final-this-time` counts as being released after `1.2.3`). - GHC package numbers are comprised of any number of numeric segments separated by `.`, plus an optional (though deprecated) alphanumeric "version tag" separated by a `-`. I could not find any official documentation on ordering for the version tag; numeric segments are compared as numbers. - npm uses semantic versioning so that is covered already. After much more investigation than I'd care to admit, I have come up with the following compromise as the least-bad solution. First, obviously, the version string for stable/marketing versions is going to be "standard" semver, i.e. major.minor.patch, all numbers, which works, and sorts as expected, for all three schemes. For snapshot releases, we shall use the following (semver) format: ``` 0.13.53-alpha.20200214.3025.ef5d32b7 ``` where the components are, respectively: - `0.13.53`: the expected version string of the next "stable" release. - `alpha`: a marker that hopefully scares people enough. - `20200214`: the date of the release commit, which _MUST_ be on master. - `3025`: the number of commits in master up to the release commit (included). Because we have a linear, append-only master branch, this uniquely identifies the commit. - `ef5d32b7ù : the first 8 characters of the release commit sha. This is not strictly speaking necessary, but makes it a lot more convenient to identify the commit. The main downsides of this format are: 1. It is not a valid format for GHC packages. We do not publish GHC packages from the SDK (so far we have instead opted to release our Haskell code as separate packages entirely), so this should not be an issue. However, our SDK version currently leaks to `ghc-pkg` as the version string for the stdlib (and prim) packages. This PR addresses that by tweaking the compiler to remove the offending bits, so `ghc-pkg` would see the above version number as `0.13.53.20200214.3025`, which should be enough to uniquely identify it. Note that, as far as I could find out, this number would never be exposed to users. 2. It is rather long, which I think is good from a human perspective as it makes it more scary. However, I have been told that this may be long enough to cause issues on Windows by pushing us past the max path size limitation of that "OS". I suggest we try it and see what happens. The upsides are: - It clearly indicates it is an unstable release (`alpha`). - It clearly indicates how old it is, by including the date. - To humans, it is immediately obvious which version is "later" even if they have the same date, allowing us to release same-day patches if needed. (Note: that is, commits that were made on the same day; the release date itself is irrelevant here.) - It contains the git sha so the commit built for that release is immediately obvious. - It sorts correctly under all schemes (modulo the modification for GHC). Alternatives I considered: - Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would be accepted by all schemes, but will not sort as expected under semantic versioning (though Maven will be fine). I have no idea how it will sort under GHC. - Not having any non-numeric component, e.g. `0.13.53.20200214.3025`. This is not valid semantic versioning and is therefore rejected by npm. - Not having detailed info: just go with `0.13.53-snapshot`. This is what is generally done in the Java world, but we then lose track of what version is actually in use and I'm concerned about bug reports. This would also not let us publish to the main Maven repo (at least not more than once), as artifacts there are supposed to be immutable. - No having a qualifier: `0.13.53-3025` would be acceptable to all three version formats. However, it would not clearly indicate to humans that it is not meant as a stable version, and would sort differently under semantic versioning (which counts it as a prerelease, i.e. before `0.13.53`) than under maven (which counts it as a patch, so after `0.13.53`). - Just counting releases: `0.13.53-alpha.1`, where we just count the number of prereleases in-between `0.13.52` and the next. This is currently the fallback plan if Windows path length causes issues. It would be less convenient to map releases to commits, but it could still be done via querying the history of the `LATEST` file. Release notes ============= > Note: We have decided not to have release notes for snapshot releases. Release notes are a bit tricky. Because we want the ability to make snapshot releases, then later on promote them to stable releases, it follows that we want to build commits from the past. However, if we decide post-hoc that a commit is actually a good candidate for a release, there is no way that commit can have the appropriate release notes: it cannot know what version number it's getting, and, moreover, we now track changes in commit messages. And I do not think anyone wants to go back to the release notes file being a merge bottleneck. But release notes need to be published to the releases blog upon releasing a stable version, and the docs website needs to be updated and include them. The only sensible solution here is to pick up the release notes as of the commit that triggers the release. As the docs cron runs asynchronously, this means walking down the git history to find the relevant commit. > Note: We could probably do away with the asynchronicity at this point. > It was originally included to cover for the possibility of a release > failing. If we are releasing commits from the past after they have been > tested, this should not be an issue anymore. If the docs generation were > part of the synchronous release step, it would have direct access to the > correct release notes without having to walk down the git history. > > However, I think it is more prudent to keep this change as a future step, > after we're confident the new release scheme does indeed produce much more > reliable "stable" releases. New release process =================== Just like releases are currently controlled mostly by detecting changes to the `VERSION` file, the new process will be controlled by detecting changes to the `LATEST` file. The format of that file will include both the version string and the corresponding SHA. Upon detecting a change to the `LATEST` file, CI will run the entire release process, just like it does now with the VERSION file. The main differences are: 1. Before running the release step, CI will checkout the commit specified in the LATEST file. This requires separating the release step from the build step, which in my opinion is cleaner anyway. 2. The `//:VERSION` Bazel target is replaced by a repository rule that gets the version to build from an environment variable, with a default of `0.0.0` to remain consistent with the current `daml-head` behaviour. Some of the manual steps will need to be skipped for a snapshot release. See amended `release/RELEASE.md` in this commit for details. The main caveat of this approach is that the official release will be a different binary from the corresponding snapshot. It will have been built from the same source, but with a different version string. This is somewhat mitigated by Bazel caching, meaning any build step that does not depend on the version string should use the cache and produce identical results. I do not think this can be avoided when our artifact includes its own version number. I must note, though, that while going through the changes required after removing the `VERSION` file, I have been quite surprised at the sheer number of things that actually depend on the SDK version number. I believe we should look into reducing that over time. CHANGELOG_BEGIN CHANGELOG_END
2020-02-25 19:01:23 +03:00
load("@build_environment//:configuration.bzl", "mvn_version")
2019-04-04 11:33:38 +03:00
alias(
name = "sandbox",
actual = "sandbox-ce",
visibility = ["//visibility:public"],
)
[
da_scala_library(
name = "sandbox-{}".format(edition),
srcs = glob(["src/main/scala/**/*.scala"]),
# Do not include logback.xml into the library: let the user
# of the sandbox-as-a-library decide how to log.
resources = ["//ledger/sandbox-common:src/main/resources/banner.txt"],
scala_deps = [
"@maven//:com_github_scopt_scopt",
"@maven//:com_typesafe_akka_akka_actor",
"@maven//:com_typesafe_akka_akka_stream",
"@maven//:org_scala_lang_modules_scala_java8_compat",
"@maven//:org_scalaz_scalaz_core",
],
tags = ["maven_coordinates=com.daml:sandbox:__VERSION__"],
visibility = [
"//visibility:public",
],
runtime_deps = [
"@maven//:ch_qos_logback_logback_classic",
"@maven//:ch_qos_logback_logback_core",
"@maven//:com_h2database_h2",
"@maven//:org_postgresql_postgresql",
"@maven//:org_xerial_sqlite_jdbc",
],
deps = [
"//daml-lf/archive:daml_lf_archive_reader",
"//daml-lf/archive:daml_lf_dev_archive_proto_java",
"//daml-lf/data",
"//daml-lf/engine",
"//daml-lf/language",
"//daml-lf/transaction",
"//language-support/scala/bindings",
"//ledger/caching",
"//ledger/ledger-api-auth",
"//ledger/ledger-api-common",
"//ledger/ledger-api-domain",
"//ledger/ledger-api-health",
"//ledger/ledger-configuration",
"//ledger/ledger-on-sql",
"//ledger/ledger-resources",
"//ledger/metrics",
"//ledger/participant-integration-api",
"//ledger/participant-state",
"//ledger/participant-state-metrics",
"//ledger/participant-state/kvutils",
"//ledger/sandbox-common:sandbox-common-{}".format(edition),
"//ledger-service/cli-opts",
"//libs-scala/build-info",
"//libs-scala/contextualized-logging",
"//libs-scala/ports",
"//libs-scala/resources",
"//libs-scala/resources-akka",
"//libs-scala/resources-grpc",
"@maven//:ch_qos_logback_logback_classic",
"@maven//:com_typesafe_config",
"@maven//:org_slf4j_slf4j_api",
],
)
for edition in [
"ce",
"ee",
]
]
da_scala_binary(
name = "sandbox-binary",
main_class = "com.daml.platform.sandboxnext.Main",
resources = ["//ledger/sandbox-common:src/main/resources/logback.xml"],
visibility = ["//visibility:public"],
Remove unused dependencies to da_scala_library (#3938) * Inline all scala_library dependencies * Run //:buildifier-fix * TMP scala_library_suite --> scala_library * da_scala_library: Enable unused dependency checker * scala_library: Enable unused dependency checker * //daml-lf/data:data * //daml-lf/engine:engine * //ledger-api/rs-grpc-akka:rs-grpc-akka * //ledger/participant-state:participant-state * //ledger/ledger-api-client:ledger-api-client * //scala-protoc-plugins/scala-logging:scala-logging-lib * //ledger/ledger-api-scala-logging:ledger-api-scala-logging * //ledger/ledger-api-common:ledger-api-common * //ledger-service/utils:utils * //ledger-service/jwt:jwt * //ledger/ledger-api-auth:ledger-api-auth * //extractor:extractor * //daml-assistant/scala-daml-project-config:scala-daml-project-config * //language-support/codegen-common:codegen-common * //language-support/scala/codegen:codegen * //language-support/codegen-main:codegen-main-lib * //ledger-service/db-backend:db-backend * //ledger-service/http-json:http-json * //daml-lf/scenario-interpreter:scenario-interpreter * //ledger/sandbox:sandbox * //navigator/backend:navigator-library * //daml-assistant/daml-sdk:sdk-lib * //daml-lf/data-scalacheck:data-scalacheck * //daml-script/test:test-lib * //ledger/ledger-api-common:ledger-api-common-scala-tests-lib * //ledger/test-common:test-common * //ledger/sandbox:sandbox-scala-tests-lib * //extractor:extractor-scala-tests-lib * //language-support/java/bindings:bindings-java-tests-lib * //language-support/java/bindings-rxjava:bindings-java-tests-lib * //language-support/scala/bindings-akka-testing:bindings-akka-testing * //language-support/scala/codegen-testing:codegen-testing * //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app * //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app-testing * //language-support/scala/codegen-testing:codegen-testing-testing * //ledger-api/sample-service:sample-service * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib * //ledger/ledger-api-test-tool:ledger-api-test-tool-lib * //ledger/ledger-api-test-tool:ledger-api-test-tool-tests * //ledger/participant-state/kvutils:kvutils * //ledger/sandbox:ledger-api-server * //ledger/sandbox-perf:sandbox-perf-lib * //navigator/backend:navigator-tests-library * UNDO scala_library_suite --> scala_library This reverts commit ab3eb1ae23139e2ec431ab4551fbb0371e0354e1. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 20:14:21 +03:00
deps = [
":sandbox",
],
2019-04-04 11:33:38 +03:00
)
da_scala_binary(
name = "sandbox-ephemeral-postgresql",
main_class = "com.daml.platform.sandboxnext.persistence.MainWithEphemeralPostgresql",
resources = ["//ledger/sandbox-common:src/main/resources/logback.xml"],
New Ledger API test tool (#2282) * New acceptance test tool prototype * Fix transaction filter helper * Address https://github.com/digital-asset/daml/pull/2282#discussion_r308081386 * Address https://github.com/digital-asset/daml/pull/2282#discussion_r308082432 Moreover, creates specific helpers for test DAML code. * Move helpers from session to context Remove a circular dependency between session and context, and limits the usage of implicits. * Run tests with ledgers out of process * Prepare for drop in * Temporary timeout increase * Move bulk of the logic to the test runner * Have a map of tests for CLI integration * Timeout per-test, runner uses context, renaming * Allow usage of time service, integrate semantic tests * Integrate CLI SSL configuration * Integrate failure expectation CLI option * Integrate with failure output verbosity CLI option * Integrate with test listing and picking CLI options * Integrate with timeout scaling CLI option * Integrate with command TTL CLI option * Review CODEOWNERS * Drop previously unimplemented CLI parameters * Expand test todo list to encompass latest additions * Deduplicate some testing infrastructure, swap in new test tool * Remove unused rule * Address https://github.com/digital-asset/daml/pull/2282#discussion_r311045444 * Address https://github.com/digital-asset/daml/pull/2282#discussion_r311051974 * Address https://github.com/digital-asset/daml/pull/2282#discussion_r311035287 * Address https://github.com/digital-asset/daml/pull/2282#discussion_r311055290 * WIP * Make gRPC thread factory spawn non-deamon threads * Remove debug prints, complete divulgence test (missing assertion messages) * Try to detect assertion failure line number * Add Sandbox binary that starts an ephemeral postgres instance This allows us to use the client_server_test bazel macro to run against the sandbox backed by postgres. Fixes #1543 * Add Sandbox binary that starts an ephemeral postgres instance This allows us to use the client_server_test bazel macro to run against the sandbox backed by postgres. Fixes #1543 * Address https://github.com/digital-asset/daml/pull/2282#discussion_r311400695 * Remove TlsConfiguration duplicate, move original to commons * Make assertion more readable Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * Fix build errors introduced in previous commit - exclude time service tests from reference-v2 - add copyright header to TlsConfiguration * Complete divulgence test and address review comments Restores comments from original divulgence tests and strives to be as close to them as possible. - Address https://github.com/digital-asset/daml/pull/2282#discussion_r311429169 - Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431156 - Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431913 * Delete old divulgence tests * Restore methods to fetch contracts by template identifier * Run conformance tests on in-memory sandbox * Run conformance tests on the persistent sandbox * Remove workflow identifier workaround * New contract keys test (#2452) * Set channel type for LedgerSession * Add ContractKeysIT * Run tests against reference-server v2 * Removed the old ContractKeysIT. However, ... ... I cannot remove ContractKeysChecks completely, because we have another test path for DARs created prior to DAML-LF 1.6, which doesn't have the submitter == maintainer restriction for contract keys. * Use codegen instead of custom code * Add missing copyright headers * Drop sandbox conformance test suite (seems to not work on Windows CI) * Make time simpler * Test tool tests to run only semantic tests * Make CI behave more consistently * Add missing files to run tests on Postrgres * Fix flaky divulgence test, introduce readable identifiers * Increase sandbox conformance tests timeout * Address https://github.com/digital-asset/daml/pull/2282#discussion_r312363246 * Reintroduce test resource extraction * Read semantic tests from the classpath rather than Bazel * Use codegen types for ContractKeys integration test allocateParty returns Primitive.Party and create and exercise also accept Primitive.Party. This way we don't have to keep a String and Primitive.Party representation around. * Bump timeout for sandbox conformance tests * Remove dependency on Bazel runfiles
2019-08-09 18:00:37 +03:00
visibility = ["//visibility:public"],
deps = [
":sandbox",
":sandbox-scala-tests-lib",
],
)
2019-04-04 11:33:38 +03:00
da_scala_library(
name = "sandbox-scala-tests-lib",
srcs = glob(["src/test/lib/**/*.scala"]),
scala_deps = [
"@maven//:com_typesafe_akka_akka_actor",
"@maven//:com_typesafe_akka_akka_stream",
"@maven//:org_scalactic_scalactic",
"@maven//:org_scalatest_scalatest_core",
"@maven//:org_scalaz_scalaz_core",
],
scala_runtime_deps = [
"@maven//:com_typesafe_akka_akka_slf4j",
],
visibility = ["//visibility:public"],
Remove unused dependencies to da_scala_library (#3938) * Inline all scala_library dependencies * Run //:buildifier-fix * TMP scala_library_suite --> scala_library * da_scala_library: Enable unused dependency checker * scala_library: Enable unused dependency checker * //daml-lf/data:data * //daml-lf/engine:engine * //ledger-api/rs-grpc-akka:rs-grpc-akka * //ledger/participant-state:participant-state * //ledger/ledger-api-client:ledger-api-client * //scala-protoc-plugins/scala-logging:scala-logging-lib * //ledger/ledger-api-scala-logging:ledger-api-scala-logging * //ledger/ledger-api-common:ledger-api-common * //ledger-service/utils:utils * //ledger-service/jwt:jwt * //ledger/ledger-api-auth:ledger-api-auth * //extractor:extractor * //daml-assistant/scala-daml-project-config:scala-daml-project-config * //language-support/codegen-common:codegen-common * //language-support/scala/codegen:codegen * //language-support/codegen-main:codegen-main-lib * //ledger-service/db-backend:db-backend * //ledger-service/http-json:http-json * //daml-lf/scenario-interpreter:scenario-interpreter * //ledger/sandbox:sandbox * //navigator/backend:navigator-library * //daml-assistant/daml-sdk:sdk-lib * //daml-lf/data-scalacheck:data-scalacheck * //daml-script/test:test-lib * //ledger/ledger-api-common:ledger-api-common-scala-tests-lib * //ledger/test-common:test-common * //ledger/sandbox:sandbox-scala-tests-lib * //extractor:extractor-scala-tests-lib * //language-support/java/bindings:bindings-java-tests-lib * //language-support/java/bindings-rxjava:bindings-java-tests-lib * //language-support/scala/bindings-akka-testing:bindings-akka-testing * //language-support/scala/codegen-testing:codegen-testing * //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app * //language-support/scala/codegen-sample-app:daml-lf-codegen-sample-app-testing * //language-support/scala/codegen-testing:codegen-testing-testing * //ledger-api/sample-service:sample-service * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib * //ledger/ledger-api-test-tool:ledger-api-test-tool-lib * //ledger/ledger-api-test-tool:ledger-api-test-tool-tests * //ledger/participant-state/kvutils:kvutils * //ledger/sandbox:ledger-api-server * //ledger/sandbox-perf:sandbox-perf-lib * //navigator/backend:navigator-tests-library * UNDO scala_library_suite --> scala_library This reverts commit ab3eb1ae23139e2ec431ab4551fbb0371e0354e1. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2020-01-06 20:14:21 +03:00
runtime_deps = [
"@maven//:com_h2database_h2",
"@maven//:org_postgresql_postgresql",
],
Remove unused dependencies to da_scala_test(_suite) (#3925) * Remove unused scala.bzl imports * override_targets org.scalatest.scalatest_2.12 Otherwise, rules_scala implicitly adds a different version to scala_test than other packages transitively depending on scalatest. This causes unused dependency checker to raise an error. * Handle rules_scala scalatest in pom_file.bzl * Inline all scala_test dependencies So that `unused_dependency_checker = "error"` can be applied to them. * Run //:buildifier-fix * TMP scala_test_suite --> scala_test * da_scala_test: Enable unused dependency checker * //navigator/backend:navigator-scala-tests * //ledger/sandbox:sandbox-scala-tests * //ledger/participant-state/kvutils:kvutils-tests * //ledger/participant-state:participant-state-tests * //ledger/ledger-api-scala-logging:ledger-api-scala-logging-test * //ledger/ledger-api-common:ledger-api-common-scala-tests * //ledger/ledger-api-client:ledger-api-client-tests * //ledger/ledger-api-auth:ledger-api-auth-scala-tests * //ledger-service/lf-value-json:tests * //ledger-service/jwt:tests * //ledger-service/http-json:tests * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests * //language-support/scala/codegen-sample-app:tests * //language-support/scala/codegen-sample-app:ScalaCodeGenIT * //language-support/scala/codegen:tests * //language-support/scala/bindings-akka:tests * //language-support/java/codegen:test * //language-support/java/codegen:ledger-tests * //language-support/java/bindings-rxjava:bindings-java-tests * //language-support/codegen-common:test * //extractor:extractor-scala-tests * //daml-lf/scenario-interpreter:scenario-interpreter_tests * //daml-lf/language:language-test * //daml-lf/interface:tests * //daml-lf/engine:tests * //daml-lf/encoder:tests * //daml-lf/archive:daml_lf_archive_reader_tests * //daml-assistant/scala-daml-project-config:scala-daml-project-config-tests * UNDO scala_test_suite --> scala_test This reverts commit 13ed47ba725e944533ca1157a070cb8dc30569ac. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2019-12-30 16:49:59 +03:00
deps = [
":sandbox",
"//daml-lf/data",
"//language-support/scala/bindings",
"//ledger-api/rs-grpc-bridge",
"//ledger-api/testing-utils",
"//ledger/caching",
Remove unused dependencies to da_scala_test(_suite) (#3925) * Remove unused scala.bzl imports * override_targets org.scalatest.scalatest_2.12 Otherwise, rules_scala implicitly adds a different version to scala_test than other packages transitively depending on scalatest. This causes unused dependency checker to raise an error. * Handle rules_scala scalatest in pom_file.bzl * Inline all scala_test dependencies So that `unused_dependency_checker = "error"` can be applied to them. * Run //:buildifier-fix * TMP scala_test_suite --> scala_test * da_scala_test: Enable unused dependency checker * //navigator/backend:navigator-scala-tests * //ledger/sandbox:sandbox-scala-tests * //ledger/participant-state/kvutils:kvutils-tests * //ledger/participant-state:participant-state-tests * //ledger/ledger-api-scala-logging:ledger-api-scala-logging-test * //ledger/ledger-api-common:ledger-api-common-scala-tests * //ledger/ledger-api-client:ledger-api-client-tests * //ledger/ledger-api-auth:ledger-api-auth-scala-tests * //ledger-service/lf-value-json:tests * //ledger-service/jwt:tests * //ledger-service/http-json:tests * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests * //language-support/scala/codegen-sample-app:tests * //language-support/scala/codegen-sample-app:ScalaCodeGenIT * //language-support/scala/codegen:tests * //language-support/scala/bindings-akka:tests * //language-support/java/codegen:test * //language-support/java/codegen:ledger-tests * //language-support/java/bindings-rxjava:bindings-java-tests * //language-support/codegen-common:test * //extractor:extractor-scala-tests * //daml-lf/scenario-interpreter:scenario-interpreter_tests * //daml-lf/language:language-test * //daml-lf/interface:tests * //daml-lf/engine:tests * //daml-lf/encoder:tests * //daml-lf/archive:daml_lf_archive_reader_tests * //daml-assistant/scala-daml-project-config:scala-daml-project-config-tests * UNDO scala_test_suite --> scala_test This reverts commit 13ed47ba725e944533ca1157a070cb8dc30569ac. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2019-12-30 16:49:59 +03:00
"//ledger/ledger-api-auth",
"//ledger/ledger-api-common",
"//ledger/ledger-api-domain",
"//ledger/ledger-configuration",
resources: Customizable contexts. (#7678) * resources: Move builders into //ledger/ledger-resources. Keep the actual constructors in a trait, but instantiate it when working with ledger code. This allows us to later introduce an extra "context" type parameter to ResourceOwner. * resources-akka: Move the builders in to //ledger/ledger-resources. * resources: Introduce an abstract `Context` parameter for owners. This replaces the concrete `ExecutionContext`. While it _can_ be an execution context, it really doesn't matter as long as we can get at one somehow. This is being introduced so we can wrap the context in a container, either for type tagging or to include extra information. Because our current context _is_ `ExecutionContext`, and an implicit is provided to extract it, we can end up with two ways to get the same value. We use shadowing to prevent this. This problem should go away in the near future when a new context type is added. CHANGELOG_BEGIN - [Integration Kit] The `ResourceOwner` type is now parameterized by a `Context`, which is filled in by the corresponding `Context` class in the _ledger-resources_ dependency. This allows us to pass extra information through resource acquisition. CHANGELOG_END * ledger-resources: Move `ResourceOwner` here from `resources`. * ledger-resources: Remove dependencies from outside //ledger. * ledger-resource: Wrap the acquisition execution context in `Context`. So we can add a logging context to it. * resources: Pass the Context, not the ExecutionContext, to Resource. * Avoid importing `HasExecutionContext`. * ledger-resources: Publish to Maven Central. * resources: Make the small changes suggested by @stefanobaghino-da. Co-Authored-By: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com> * ledger-resources: Pull out a trait for test resource contexts. Saves a few lines of code. * Restore some imports that were accidentally wildcarded. * resources: Replace an `implicit def` with a couple of imports. * participant-integration-api: Simplify the JdbcLedgerDaoBackend tests. Try and use the right execution context where possible. Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2020-10-20 12:26:28 +03:00
"//ledger/ledger-resources",
"//ledger/participant-integration-api",
"//ledger/participant-integration-api:participant-integration-api-tests-lib",
"//ledger/sandbox-common",
"//ledger/sandbox-common:sandbox-common-scala-tests-lib",
"//libs-scala/ports",
"//libs-scala/postgresql-testing",
"//libs-scala/resources",
Remove unused dependencies to da_scala_test(_suite) (#3925) * Remove unused scala.bzl imports * override_targets org.scalatest.scalatest_2.12 Otherwise, rules_scala implicitly adds a different version to scala_test than other packages transitively depending on scalatest. This causes unused dependency checker to raise an error. * Handle rules_scala scalatest in pom_file.bzl * Inline all scala_test dependencies So that `unused_dependency_checker = "error"` can be applied to them. * Run //:buildifier-fix * TMP scala_test_suite --> scala_test * da_scala_test: Enable unused dependency checker * //navigator/backend:navigator-scala-tests * //ledger/sandbox:sandbox-scala-tests * //ledger/participant-state/kvutils:kvutils-tests * //ledger/participant-state:participant-state-tests * //ledger/ledger-api-scala-logging:ledger-api-scala-logging-test * //ledger/ledger-api-common:ledger-api-common-scala-tests * //ledger/ledger-api-client:ledger-api-client-tests * //ledger/ledger-api-auth:ledger-api-auth-scala-tests * //ledger-service/lf-value-json:tests * //ledger-service/jwt:tests * //ledger-service/http-json:tests * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests * //language-support/scala/codegen-sample-app:tests * //language-support/scala/codegen-sample-app:ScalaCodeGenIT * //language-support/scala/codegen:tests * //language-support/scala/bindings-akka:tests * //language-support/java/codegen:test * //language-support/java/codegen:ledger-tests * //language-support/java/bindings-rxjava:bindings-java-tests * //language-support/codegen-common:test * //extractor:extractor-scala-tests * //daml-lf/scenario-interpreter:scenario-interpreter_tests * //daml-lf/language:language-test * //daml-lf/interface:tests * //daml-lf/engine:tests * //daml-lf/encoder:tests * //daml-lf/archive:daml_lf_archive_reader_tests * //daml-assistant/scala-daml-project-config:scala-daml-project-config-tests * UNDO scala_test_suite --> scala_test This reverts commit 13ed47ba725e944533ca1157a070cb8dc30569ac. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2019-12-30 16:49:59 +03:00
"@maven//:ch_qos_logback_logback_classic",
"@maven//:io_netty_netty_handler",
"@maven//:org_scalatest_scalatest_compatible",
Remove unused dependencies to da_scala_test(_suite) (#3925) * Remove unused scala.bzl imports * override_targets org.scalatest.scalatest_2.12 Otherwise, rules_scala implicitly adds a different version to scala_test than other packages transitively depending on scalatest. This causes unused dependency checker to raise an error. * Handle rules_scala scalatest in pom_file.bzl * Inline all scala_test dependencies So that `unused_dependency_checker = "error"` can be applied to them. * Run //:buildifier-fix * TMP scala_test_suite --> scala_test * da_scala_test: Enable unused dependency checker * //navigator/backend:navigator-scala-tests * //ledger/sandbox:sandbox-scala-tests * //ledger/participant-state/kvutils:kvutils-tests * //ledger/participant-state:participant-state-tests * //ledger/ledger-api-scala-logging:ledger-api-scala-logging-test * //ledger/ledger-api-common:ledger-api-common-scala-tests * //ledger/ledger-api-client:ledger-api-client-tests * //ledger/ledger-api-auth:ledger-api-auth-scala-tests * //ledger-service/lf-value-json:tests * //ledger-service/jwt:tests * //ledger-service/http-json:tests * //ledger-api/rs-grpc-akka:rs-grpc-akka-tests * //language-support/scala/codegen-sample-app:tests * //language-support/scala/codegen-sample-app:ScalaCodeGenIT * //language-support/scala/codegen:tests * //language-support/scala/bindings-akka:tests * //language-support/java/codegen:test * //language-support/java/codegen:ledger-tests * //language-support/java/bindings-rxjava:bindings-java-tests * //language-support/codegen-common:test * //extractor:extractor-scala-tests * //daml-lf/scenario-interpreter:scenario-interpreter_tests * //daml-lf/language:language-test * //daml-lf/interface:tests * //daml-lf/engine:tests * //daml-lf/encoder:tests * //daml-lf/archive:daml_lf_archive_reader_tests * //daml-assistant/scala-daml-project-config:scala-daml-project-config-tests * UNDO scala_test_suite --> scala_test This reverts commit 13ed47ba725e944533ca1157a070cb8dc30569ac. Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
2019-12-30 16:49:59 +03:00
"@maven//:org_slf4j_slf4j_api",
],
2019-04-04 11:33:38 +03:00
)
openssl_executable = "@openssl_dev_env//:bin/openssl" if not is_windows else "@openssl_dev_env//:usr/bin/openssl.exe"
# The reset service is cursed so we mark all tests involving it as flaky.
reset_service_pattern = "src/test/suite/**/*ResetService*.scala"
[
da_scala_test_suite(
name = "sandbox-tests{}".format(suffix),
srcs = glob(
[pattern],
exclude = exclusions,
),
data = [
"//ledger/test-common:dar-files",
"//ledger/test-common/test-certificates",
openssl_executable,
],
flaky = flaky,
jvm_flags = [
"-Djava.security.debug=\"certpath ocsp\"", # This facilitates debugging of the OCSP checks mechanism
],
resources = glob(["src/test/resources/**/*"]),
scala_deps = [
"@maven//:com_typesafe_akka_akka_actor",
"@maven//:com_typesafe_akka_akka_stream",
"@maven//:com_typesafe_akka_akka_stream_testkit",
"@maven//:com_typesafe_akka_akka_testkit",
"@maven//:org_mockito_mockito_scala",
"@maven//:org_scalacheck_scalacheck",
"@maven//:org_scalactic_scalactic",
"@maven//:org_scalatest_scalatest_core",
"@maven//:org_scalatest_scalatest_matchers_core",
"@maven//:org_scalatest_scalatest_shouldmatchers",
"@maven//:org_scalatest_scalatest_wordspec",
"@maven//:org_scalaz_scalaz_core",
],
deps = [
":sandbox",
":sandbox-scala-tests-lib",
"//bazel_tools/runfiles:scala_runfiles",
"//daml-lf/archive:daml_lf_archive_reader",
"//daml-lf/archive:daml_lf_dev_archive_proto_java",
"//daml-lf/data",
"//daml-lf/engine",
"//daml-lf/interpreter",
"//daml-lf/language",
"//daml-lf/transaction",
"//daml-lf/transaction-test-lib",
"//language-support/scala/bindings",
"//ledger-api/rs-grpc-akka",
"//ledger-api/rs-grpc-akka:rs-grpc-akka-tests-lib",
"//ledger-api/rs-grpc-bridge",
"//ledger-api/sample-service",
"//ledger-api/testing-utils",
"//ledger/caching",
"//ledger/ledger-api-auth",
"//ledger/ledger-api-client",
"//ledger/ledger-api-common",
"//ledger/ledger-api-common:ledger-api-common-scala-tests-lib",
"//ledger/ledger-api-domain",
"//ledger/ledger-api-health",
"//ledger/ledger-configuration",
"//ledger/ledger-offset",
"//ledger/ledger-resources",
"//ledger/ledger-resources:ledger-resources-test-lib",
"//ledger/metrics",
"//ledger/participant-integration-api",
"//ledger/participant-integration-api:participant-integration-api-tests-lib",
"//ledger/participant-state",
"//ledger/participant-state-index",
"//ledger/participant-state-metrics",
"//ledger/sandbox-common",
"//ledger/sandbox-common:sandbox-common-scala-tests-lib",
"//ledger/test-common",
"//libs-scala/concurrent",
"//libs-scala/contextualized-logging",
"//libs-scala/grpc-utils",
"//libs-scala/ports",
"//libs-scala/postgresql-testing",
"//libs-scala/resources",
"@maven//:ch_qos_logback_logback_classic",
"@maven//:ch_qos_logback_logback_core",
"@maven//:com_typesafe_config",
"@maven//:commons_io_commons_io",
"@maven//:io_dropwizard_metrics_metrics_core",
"@maven//:io_grpc_grpc_services",
"@maven//:io_netty_netty_handler",
"@maven//:org_awaitility_awaitility",
"@maven//:org_flywaydb_flyway_core",
"@maven//:org_mockito_mockito_core",
"@maven//:org_reactivestreams_reactive_streams",
"@maven//:org_scalatest_scalatest_compatible",
"@maven//:org_slf4j_slf4j_api",
],
)
for (suffix, pattern, exclusions, flaky) in [
(
"",
"src/test/suite/**/*.scala",
[reset_service_pattern],
False,
),
(
"-resetservice",
reset_service_pattern,
[],
True,
),
]
]
NEXT_SERVERS = {
"memory": {
"binary": ":sandbox-binary",
"server_args": [
"--contract-id-seeding=testing-weak",
"--port=6865",
"--eager-package-loading",
],
},
"postgresql": {
"binary": ":sandbox-ephemeral-postgresql",
"server_args": [
"--contract-id-seeding=testing-weak",
"--port=6865",
"--eager-package-loading",
],
},
}
server_conformance_test(
name = "next-conformance-test-static-time",
lf_versions = [
"default",
"latest",
"preview",
],
server_args = [
"--static-time",
],
servers = NEXT_SERVERS,
test_tool_args = [
"--open-world",
"--exclude=CommandDeduplicationIT", # It's a KV ledger so it needs the KV variant
"--exclude=ClosedWorldIT",
],
)
server_conformance_test(
name = "next-conformance-test-wall-clock",
server_args = [
"--wall-clock-time",
"--max-deduplication-duration=PT5S",
],
servers = NEXT_SERVERS,
test_tool_args = [
"--open-world",
"--additional=KVCommandDeduplicationIT",
"--exclude=CommandDeduplicationIT", # It's a KV ledger so it needs the KV variant
"--exclude=ClosedWorldIT",
],
)
server_conformance_test(
name = "next-conformance-test-closed-world",
server_args = [
"--wall-clock-time",
"--implicit-party-allocation=false",
],
servers = NEXT_SERVERS,
test_tool_args = [
"--verbose",
"--include=ClosedWorldIT",
],
)
server_conformance_test(
name = "next-conformance-test-contract-ids",
servers = {"memory": NEXT_SERVERS["memory"]},
test_tool_args = [
"--verbose",
"--include=ContractIdIT:RejectV0,ContractIdIT:AcceptSuffixedV1,ContractIdIT:AcceptNonSuffixedV1",
],
)
# TODO append-only: cleanup
server_conformance_test(
name = "next-conformance-test-append-only",
server_args = [
"--enable-append-only-schema",
"--max-deduplication-duration=PT5S",
],
servers = {"postgresql": NEXT_SERVERS["postgresql"]},
test_tool_args = [
"--open-world",
"--additional=AppendOnlyCompletionDeduplicationInfoITCommandService",
"--additional=AppendOnlyCompletionDeduplicationInfoITCommandSubmissionService",
"--additional=AppendOnlyKVCommandDeduplicationIT",
"--exclude=CommandDeduplicationIT", # It's a KV ledger so it needs the KV variant
"--exclude=ClosedWorldIT",
],
)