* reduce fork count, measurement and warmup iterations, and extra parameters which had a multiplicative effect on total work done
* fix db connection setup - the combination of annotations and inheritance meant it was trying to setup the trial twice, causing Postgres benchmarks to fail
* log when an error occurs during test setup
* add logback resources to the benchmarks, to enable configuration of logging, and avoid dumping all the debug logs by default
With these changes, on my 20 core 64G linux laptop, the benchmarks now all run in under 6 mins.
Without this change, it took over 9 hours in total and none of the Postgres benchmarks were successful.
The securityTestLayer property used to be defined on a per test suite basis. The value may vary from test case to test case. Therefore, it is not always possible to assign a correct value. Removing the property to resolve this problem.
CHANGELOG_BEGIN
CHANGELOG_END
Adds a `disclosedContracts` optional list field to the `meta` argument
for `create`, `exercise` and `create-and-exercise` endpoints.
The argument is ignored in all cases but `exercise` (#16611 builds on
this PR to add `create-and-exercise` support). A single disclosed
contract looks more or less like follows:
{
"contractId": "abcd",
"templateId": "Mod:Tmpl",
$argumentsJsonField,
"metadata": {
"createdAt": "2023-03-21T18:00:33.246813Z",
"contractKeyHash": "77656c6c2068656c6c6f",
"driverMetadata": "dGhlcmUgcmVhZGVy"
}
}
where `argumentsJsonField` may be either one of these, setting aside the
extra quotes added for these tests:
"payload": {"owner": "Alice"}
"payloadBlob": {
"typeUrl": "type.googleapis.com/com.daml.ledger.api.v1.Record",
"value": "Eg4KBW93bmVyEgVaA0JvYg=="
}
(Note that `typeUrl` is variable, not constant; use the actual blob's
`typeUrl` contents, **do not assume it is exactly the above example**.)
This PR uses base-64 for `payloadBlob.value` and
`metadata.driverMetadata`, and base-16 for `metadata.contractKeyHash`.
After this PR the TimerResourceOwner will make sure that none of
the scheduled task are running, after the Resource finished releasing.
Also adds asynchronous test as evidence.
[CHANGELOG_BEGIN]
[CHANGELOG_END]
* confirm that monadifying the package fetch still suppresses the error
* thread ExecutionContext from request
- makes the error less likely
- but still fairly easy to repro with 3 tabs
* experiment with setting executor
* explain that the cache isn't a cache
* random order, maybe
- #3090 mentions keeping the order as a goal; I don't see why we should,
though
* random order with groups of 8
* embed the decoding
- this slows down the processing of a group, yielding somewhat less
granular contention
- and also makes hits cost much less, at the cost of making granular
contention more expensive
* reduce diff size before resolution
- this won't improve contention, but does nearly eliminate the cost of
resolution for already-resolved packages, making hits nearly free
(amortized)
* randomize groups instead
- while groups themselves can overlap with this arrangement, each
costing ParallelLoadFactor granular contention, on average it seems to
perform a little better due to groups never overlapping
* refactor StatusEnvelope to utils
* constant 250ms retry
* detect contention earlier and skip decode
* factor traverseFM
* declare needed NonEmpty query lists and condition lists
* selectContractsMultiTemplate requires non-empty query list
* propagate nonempty query sets through ContractDao
* propagate some NE constraints from selectContractsMultiTemplate through WebSocketService
* HashSet no longer needed
* pass non-emptiness through dbQueries
* add NE-preserving groupMap and groupMap1
* validate that resolvedWithKey is nonempty
* add -Xlint options requiring no changes
* add -Xlint:recurse-with-default
- very minor code changes
* factor http-json hj_scalacopts duplication
* use lf_scalacopts_stricter in libs-scala where NonUnitStatements was
* use hj_scalacopts in api-type-signature
* add nonlocal-return and nullary-unit to hj_scalacopts
* commented-out excluded options
* add unit-special globally
* check implicit-recursion for clients code
It turned out that throwing an exception is too strict for the downstream Canton project
which currently fails on CI caused by errors in a number of test suites.
Until they have adapted (see https://github.com/DACH-NY/canton/issues/11083) we only warn about
the errors by default.
For this project we ensure that warnings are fatal.
* Add new security categories
* Use test-security framework for OAuth2 middleware
* Regenerate `security-evidence.md`
* Explicitely exit the test-evidence generator
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Add //ledger-service/http-json:integration-tests-lib to evidence generator
* Skip maven artefacts on the classpath
The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
at scala.collection.immutable.List.prependedAll(List.scala:152)
at scala.collection.immutable.List$.from(List.scala:684)
at scala.collection.immutable.List$.from(List.scala:681)
at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
at scala.collection.immutable.List.map(List.scala:250)
```
* Regenerate `security-evidence.md`
* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Print warning when a test suite could not be loaded
* Fix typo
* Use test-security framework for HTTP JSON
* Read files in tests lazily
The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for trigger service
* Use structural type to call `in` and `ignore` for different classes
* Remove Authentication category from EvidenceSecurity tool
There are no TEST_EVIDENCE annotations anymore.
* Add required trigger-runner-lib
* Import `scala.language.reflectiveCalls` where it is needed
* Remove left-over comments
* Add `test_evidence_binary` scala binary rule
This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.
* Process test suites, add transitive deps
* Support nested tests_suites in `test_evidence_binary`
* Remove debug print's
* Add missing dependencies to test-evidence:generator
* Abort if test suites cannot be loaded
* Cleanup
* Reinstate scalacopts in http-json
* Reword the test description to not drop information
* Fix typo
* Explicitly exit the JVM on exceptions
This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.
* Format test-evidence/BUILD.bazel
* Resolve file paths lazily
This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.
* Document new Security properties
* Print target directory and file name
* Clarify test descriptions
* Replace duplicate Security properties
Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Added logging for time elapsed of ledger client call
CHANGELOG_BEGIN
CHANGELOG_END
fixes#14673
* rename logging def
* ADDED TODO
* ADDED TODO
* pass in boolean for determining if it should log based on the log level
* address Ray's comment. Wrong name Ledge -? Ledger
* make isEnable public and use it to determin whether it is debug level
* refactoring. put everything in a inner object LedgerClientRequestTimeLogger
* remove useless line
* address Ray's comment
* address Ray's comment
* log the time used for source as well
* address Stephen's address
CHANGELOG_BEGIN
- [JSON API] For exercise endpoints, when multiple choices with the same
name are defined, the one directly defined on the ``templateId``,
which may be a template or interface ID, will be selected; if a
template ID is specified that only inherits the choices with the same
name, the new ``choiceInterfaceId`` option must be specified to select
the source of the choice.
CHANGELOG_END
* introduce TemplateChoices to mediate template choice resolution
* if any resolutions fail in EnvironmentInterface, discard the template's resolution
* if any resolutions fail in Interface, discard the remaining unresolved
CHANGELOG_BEGIN
CHANGELOG_END
Previously a fire and forget release process
was implemented, which in corner cases could
result in exceptions, as API stream was still running after DbDispatcher was released.
This PR changes to waiting for all the streams to finish as releasing Dispatcher (with a timeout of 5 seconds).
CHANGELOG_BEGIN
CHANGELOG_END
