daml/test-evidence/test_evidence.bzl
Claudio Bley e766f91338
Generate CSV file from all test evidences for ledger client components (#15372)
* Add new security categories

* Use test-security framework for OAuth2 middleware

* Regenerate `security-evidence.md`

* Explicitely exit the test-evidence generator

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Add //ledger-service/http-json:integration-tests-lib to evidence generator

* Skip maven artefacts on the classpath

The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
	at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
	at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
	at scala.collection.immutable.List.prependedAll(List.scala:152)
	at scala.collection.immutable.List$.from(List.scala:684)
	at scala.collection.immutable.List$.from(List.scala:681)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
	at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
	at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
	at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
	at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
	at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
	at scala.collection.immutable.List.map(List.scala:250)
```

* Regenerate `security-evidence.md`

* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Print warning when a test suite could not be loaded

* Fix typo

* Use test-security framework for HTTP JSON

* Read files in tests lazily

The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for HTTP JSON

* Regenerate `security-evidence.md`

* Use test-security framework for trigger service

* Use structural type to call `in` and `ignore` for different classes

* Remove Authentication category from EvidenceSecurity tool

There are no TEST_EVIDENCE annotations anymore.

* Add required trigger-runner-lib


* Import `scala.language.reflectiveCalls` where it is needed

* Remove left-over comments

* Add `test_evidence_binary` scala binary rule

This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.

* Process test suites, add transitive deps

* Support nested tests_suites in `test_evidence_binary`

* Remove debug print's

* Add missing dependencies to test-evidence:generator

* Abort if test suites cannot be loaded

* Cleanup

* Reinstate scalacopts in http-json

* Reword the test description to not drop information

* Fix typo

* Explicitly exit the JVM on exceptions

This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.

* Format test-evidence/BUILD.bazel

* Resolve file paths lazily

This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.

* Document new Security properties

* Print target directory and file name

* Clarify test descriptions

* Replace duplicate Security properties

Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
2022-11-29 12:35:22 +01:00

91 lines
3.0 KiB
Python

# Copyright (c) 2022 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
load("@io_bazel_rules_scala//scala:advanced_usage/providers.bzl", "ScalaRulePhase")
load("@io_bazel_rules_scala//scala:advanced_usage/scala.bzl", "make_scala_binary")
TestSuiteInfo = provider(
fields = {
"tests": "tests the suite runs",
},
)
def _test_suite_info_aspect_impl(target, ctx):
if ctx.rule.kind == "test_suite":
tests = ctx.rule.attr.tests
else:
tests = []
return [TestSuiteInfo(tests = tests)]
_test_suite_info_aspect = aspect(implementation = _test_suite_info_aspect_impl, attr_aspects = ["tests"])
_write_scalatest_runpath_phase = {
"attrs": {
"tests": attr.label_list(allow_files = True, aspects = [_test_suite_info_aspect]),
},
"phase_providers": [
"@com_github_digital_asset_daml//test-evidence:phase_write_scalatest_runpath",
],
}
def phase_write_scalatest_runpath(ctx, p):
runfiles_ext = []
test_jars = []
for target in ctx.attr.tests:
if TestSuiteInfo in target:
for suite_target in target[TestSuiteInfo].tests:
files = suite_target.files.to_list()
if files:
test_jar = files[0]
test_jars.append(test_jar.short_path)
runfiles_ext.append(test_jar)
elif TestSuiteInfo in suite_target:
for sub_target in suite_target[TestSuiteInfo].tests:
files = sub_target.files.to_list()
test_jar = files[0]
test_jars.append(test_jar.short_path)
runfiles_ext.append(test_jar)
files = target.files.to_list()
if files:
test_jars.append(files[0].short_path)
runpath_file = ctx.actions.declare_file("%s.runpath" % ctx.label.name)
runfiles_ext.append(runpath_file)
runfiles_ext.extend(ctx.files.tests)
compile_jars = [target[JavaInfo].compile_jars for target in ctx.attr.tests if JavaInfo in target]
runtime_jars = [target[JavaInfo].transitive_runtime_jars for target in ctx.attr.tests if JavaInfo in target]
for target in ctx.attr.tests:
if JavaInfo in target:
deps = target[JavaInfo].transitive_runtime_jars
ctx.actions.write(
output = runpath_file,
content = "\n".join(test_jars),
)
return struct(
runfiles = depset(runfiles_ext),
transitive_runtime_jars = depset(ctx.files.tests, transitive = runtime_jars + compile_jars),
)
test_evidence_binary = make_scala_binary(_write_scalatest_runpath_phase)
def _write_scalatest_runpath_impl(ctx):
return [
ScalaRulePhase(
custom_phases = [
("first", "", "write_scalatest_runpath_phase", phase_write_scalatest_runpath),
],
),
]
write_scalatest_runpath = rule(
implementation = _write_scalatest_runpath_impl,
)