* declare needed NonEmpty query lists and condition lists
* selectContractsMultiTemplate requires non-empty query list
* propagate nonempty query sets through ContractDao
* propagate some NE constraints from selectContractsMultiTemplate through WebSocketService
* HashSet no longer needed
* pass non-emptiness through dbQueries
* add NE-preserving groupMap and groupMap1
* validate that resolvedWithKey is nonempty
* log more termination
From the timeout loop:
+ fmm-outer
+ fmm-inner
x ACS-before-tx
x tx-after-ACS
* spam eagerCancel=true and see what happens
From the timeout loop:
+ after-split
+ IDSS-outer
+ fmm-outer
+ contractsAndBoundary
+ tx-after-ACS
+ fmm-inner
+ GTSFP-outer
x ACS-before-tx
* passing acs-and-tx tests
* trying combinations of reverting eagerCancel settings
- setting eagerCancel = false in acsAndBoundary causes the ACS
cancellation to fail (first test), but the tx cancellation still
succeeds
- setting eagerCancel = false in project2 causes both the ACS and tx
stream cancellation tests (first and third tests) to fail
- the offset broadcast in acsFollowingAndBoundary appears to be
redundant with respect to cancellation, so we revert it in the
interest of conservatism
* make test size small
* current measurement
Still fine after the refactoring of logTermination and removal of fmm-*.
+ GTSFP-outer
+ contractsAndBoundary
x IDSS-outer-2
+ after-split
+ tx-after-ACS
+ IDSS-outer-1
x ACS-before-tx
* set level of the logTermination messages to trace
* Add "component" to `SecurityTest` entry for AbstractWebsocketServiceIntegrationTest
* Add README to test-evidence explaining the convention for components
* WIP
* do it on getuesr first
* remove old getUesr
* createUser
* createUser
* user post paths
* user post paths
* WIP
* removed routesetup from UserManagement
* refactor parties and allocateParty
* remove proxyWithCommand
* refactor PackagesAndDars
* remove proxyWithoutCommand
* refactor PackagesAndDars
* merge from main
* move UploadDarFile implementation back to package and dars
* add -Xlint options requiring no changes
* add -Xlint:recurse-with-default
- very minor code changes
* factor http-json hj_scalacopts duplication
* use lf_scalacopts_stricter in libs-scala where NonUnitStatements was
* use hj_scalacopts in api-type-signature
* add nonlocal-return and nullary-unit to hj_scalacopts
* commented-out excluded options
* add unit-special globally
* check implicit-recursion for clients code
* Add new security categories
* Use test-security framework for OAuth2 middleware
* Regenerate `security-evidence.md`
* Explicitely exit the test-evidence generator
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Add //ledger-service/http-json:integration-tests-lib to evidence generator
* Skip maven artefacts on the classpath
The checker framework artefact (`checker-2.5.4.jar`) causes an `IllegalAccessError` when included in the runpath:
```
Exception in thread "main" java.lang.IllegalAccessError: class com.sun.tools.javac.code.Scope$ImportScope$ImportEntry cannot access its superclass com.sun.tools.javac.code.Scope$Entry (com.sun.tools.javac.code.Scope$ImportScope$ImportEntry is in unnamed module of loader java.net.URLClassLoader @31000e60; com.sun.tools.javac.code.Scope$Entry is in module jdk.compiler of loader app)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:550)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at org.scalatest.tools.SuiteDiscoveryHelper$.isDiscoverableSuite(SuiteDiscoveryHelper.scala:204)
at org.scalatest.tools.SuiteDiscoveryHelper$.processClassName(SuiteDiscoveryHelper.scala:243)
at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$processFileNames$1(SuiteDiscoveryHelper.scala:279)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:474)
at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:576)
at scala.collection.immutable.List.prependedAll(List.scala:152)
at scala.collection.immutable.List$.from(List.scala:684)
at scala.collection.immutable.List$.from(List.scala:681)
at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
at scala.collection.immutable.Iterable$.from(Iterable.scala:35)
at scala.collection.immutable.Iterable$.from(Iterable.scala:32)
at scala.collection.IterableFactory$Delegate.from(Factory.scala:288)
at scala.collection.IterableOnceExtensionMethods$.toIterable$extension(IterableOnce.scala:178)
at org.scalatest.tools.SuiteDiscoveryHelper$.processFileNames(SuiteDiscoveryHelper.scala:285)
at org.scalatest.tools.SuiteDiscoveryHelper$.$anonfun$discoverSuiteNames$1(SuiteDiscoveryHelper.scala:132)
at scala.collection.immutable.List.map(List.scala:250)
```
* Regenerate `security-evidence.md`
* Convert remaining TEST_EVIDENCE stanzas of HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Print warning when a test suite could not be loaded
* Fix typo
* Use test-security framework for HTTP JSON
* Read files in tests lazily
The test-evidence generator tool needs to instantiate scalatest test suites in order
to access the tagged tests and collect relavant test entries.
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for HTTP JSON
* Regenerate `security-evidence.md`
* Use test-security framework for trigger service
* Use structural type to call `in` and `ignore` for different classes
* Remove Authentication category from EvidenceSecurity tool
There are no TEST_EVIDENCE annotations anymore.
* Add required trigger-runner-lib
* Import `scala.language.reflectiveCalls` where it is needed
* Remove left-over comments
* Add `test_evidence_binary` scala binary rule
This rule is a customized `scala_binary` rule which also accepts a `tests` attr and
generates a runpath file which is later consumed by scalatest to detect the relevant
scalatest test suites.
* Process test suites, add transitive deps
* Support nested tests_suites in `test_evidence_binary`
* Remove debug print's
* Add missing dependencies to test-evidence:generator
* Abort if test suites cannot be loaded
* Cleanup
* Reinstate scalacopts in http-json
* Reword the test description to not drop information
* Fix typo
* Explicitly exit the JVM on exceptions
This is required since non-daemon threads also prevent JVM shutdown when an exception was thrown.
* Format test-evidence/BUILD.bazel
* Resolve file paths lazily
This avoids a `NullPointerException` on Windows where Runfiles.rlocation returns `null`.
* Document new Security properties
* Print target directory and file name
* Clarify test descriptions
* Replace duplicate Security properties
Co-authored-by: Stephen Compall <stephen.compall@daml.com>Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Partly because the current version is from February (of this year,
fortunately), and partly because I need a more recent one to bump node
to 18 and fix the GSG for that.
* Adds the standard metrics to the HTTP endpoint
Refactors akka http metrics support to be used by both json-api and trigger-service
CHANGELOG_BEGIN
* Trigger Service - basic and HTTP endpoint metrics
CHANGELOG_END
I think it is reasonable to match all our default date log format across our components. The conversion is simple enough, but I did it at least 10 times just in the past 2 weeks.
* Support for golden signals metrics on akka-http
Adds metrics support needed for throughput, error rate, latency,
and data size on both the classic http endpoints and
the websocket endpoints.
Installed the support on the JSON API endpoints.
CHANGELOG_BEGIN
JSON API - additional metrics for endpoints
daml_http_json_api_requests_total
daml_http_json_api_errors_total
daml_http_json_api_requests_duration_seconds
daml_http_json_api_requests_size_bytes
daml_http_json_api_responses_size_bytes
daml_http_json_api_websocket_messages_received_total
daml_http_json_api_websocket_messages_received_size_bytes
daml_http_json_api_websocket_messages_sent_total
daml_http_json_api_websocket_messages_sent_size_bytes
CHANGELOG_END
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Update contracts table to allow multiple rows per contract id
* Keep template id for deletes
* add tpid grouping to deleteContracts
* chunking implementation
* mention tpid for postgres ON CONFLICT
* Remove ByInterfaceId special case
* add changelog
CHANGELOG_BEGIN
- [JSON API] The JSON API query store database has changed schema and
needs to be reset. If you are migrating from a previous version,
either reset your database manually or start the HTTP JSON API with
one of the options that regenerate the schema (``create-only``,
``create-if-needed-and-start``, ``create-and-start``).
CHANGELOG_END
* add tpid to ignore_row_on_dupkey_index pragma
- came up in discussion with @ray-roestenburg-da; thanks
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* use domain.Party instead of String in jwtForParties, headersWithPartyAuth
CHANGELOG_BEGIN
CHANGELOG_END
* use domain.Party instead of String in headersWithPartyAuth
* fix sharedAccountCreateCommand and some corners
* move resolveTemplateId to ContractTypeIdMap
* replace the contractTypeIdMap with an interfaceIdMap
* remove ResolveContractTypeId's type parameter, replace with Overload relation
* define the ContractTypeId resolver to be overload, ctid-type-sensitive
* remove AnyKind usages
* remove resolveTemplateId argument-passing
* demonstrate very strange inserted cast
`x.packageId` inserts a cast on `x` to `ContractTypeId.Template`, for
which there is no local evidence
* fix compiler-inserted cast problem
- Why does this fix it? Who knows!
* failure of template ID resolution no longer fails later than we want
* fix resolution of interface IDs for websocket queries
CHANGELOG_BEGIN
- [JSON API] Some error messages have been corrected to no longer
confuse interface IDs with template IDs, based on what varieties of
contract type ID the endpoint in question supports.
CHANGELOG_END
* mention #15293 where relevant
* define templateId_ generically
* remove unused utilities for old resolver
* domain.ContractTypeId.Template
* ResolveKeyType to use .Template
* CreateAndExerciseCommand to use .Template
* keyTypeMap
* bitraverse
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Removed alias fetchcontracts.domain.TemplateId
CHANGELOG_BEGIN
CHANGELOG_END
* order
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* add comment for .template
* format
* todo
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* declare the separate kind of duplication
CHANGELOG_BEGIN
CHANGELOG_END
* move completely independent tests
* move many tests to 'QueryStoreIndependent'
* fix metering-report test to work with user tokens
* rename query-store/auth-dependent tests
- to QueryStoreAndAuthDependentIntegrationTest
- from AbstractHttpServiceIntegrationTestTokenIndependent
* security evidence moved
* introduce ForQuery to overload ActiveContract.fromLedgerApi
* remove booleans from ResolvedQuery.apply
* wrong error string
* use the fact that query is non-empty to hide the Empty case
* we definitely only need a Semigroup, which fixes the law violation
* remove Empty
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Chun Lok Ling <110462561+chunlokling-da@users.noreply.github.com>
* use Consume.syntax
* added exercise in interface "sub query endpoint" test case. checked template id of archive. it is asserting to be template id but not interface id which is wrong. I will change it back to interface id after all CI passed
* test that the archive events have the interface ID; add the template-ID-to-interface-ID mapping to archive events in websocket result stream
* Add change log
CHANGELOG_BEGIM
CHANGELOG_END
* Add change log
CHANGELOG_BEGIN
CHANGELOG_END
* use JSON converter instead of hand-writing JSON
* factor assertions for created records
* combine value assertions
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* replace several TemplateId with ContractTypeId
* replace several TemplateId with ContractTypeId.Template
* deprecate, even though this misses so many usages
* link to #15098 for followups
CHANGELOG_BEGIN
CHANGELOG_END
* Remove reliance on implict party allocation in the authorization tests of the sandbox-on-x
CHANGELOG_BEGIN
CHANGELOG_END
* Remove the HOCON and cli config parameters
* Remove implicit party allocation from the Submission Service
* Remove support for implicit parties from the bridge
* Fix codegen reliance on implicit party allocation
* Fix daml script tests
* Fix trigger tests
* format security-evidence.md post rebase
* Fix json-api tests
* One more json-api test
* Fix haskell tests
* Corrections post-review
* Cosmetic improvements
* Fix race condition in party allocation handling
* a couple comparison query scenarios with nested position
* same JSON null test, but different
- 'gt string' (scenario 0) fails without this change
- 'gt int' (scenario 1) succeeds regardless of this change
CHANGELOG_BEGIN
- [JSON API with Oracle] Fix some nested queries that were returning no
data. DisableContractPayloadIndexing must be set to true to take
advantage of this fix.
CHANGELOG_END
* point to #14844 and #15040 for some TODOs
* duplicate the Oracle tests, disable large literals when indexing is on
* split test file for parallelization
CHANGELOG_BEGIN
CHANGELOG_END
