* Move //ledger/ledger-api-test-tool to //ledger/ledger-api-tests/tool.
And conformance.bzl to //ledger/ledger-api-tests.
* Move test tool infra to //ledger/ledger-api-tests/infrastructure.
* Split the ledger API test tool into smaller libraries.
There is no change to behavior.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-tests: Encapsulate performance tests.
* ledger-api-tests: Extract a "runner" library.
* ledger-api-tests: Publish libraries to Maven Central.
* ledger-api-tests: Fix Scaladoc so it compiles correctly.
Unfortunately I had to remove some Scaladoc in ActiveContractsServiceIT
which is incorrectly interpreted as not being attached to anything.
Fortunately, it's not actually that helpful.
* compatibility: Fix paths to the Ledger API Test Tool.
* Fix release types for ledger API test libraries.
* test-common: Publish generated Scala.
These libraries are dependencies of the ledger API tests components.
* ledger-api-tests: Use "com.daml" as the group ID everywhere.
* ledger-api-tests: Check that tests in `--additional` are valid.
* compatibility: Fix a reference to ledger-api-test-tool I missed.
Since Scala 2.13.2, Scala introduced built-in support to
manage warnings in a more granular fashion, thus making
the silencer plugin we are currently using no longer
strictly useful. Removing compiler plugins also removes
friction from migrating to Scala 3 in the future. As a
cherry on top, the built-in warning configuration also
allows to check whether a `@nowarn` actually does
anything, allowing us to proactively remove unused
warnings should the need arise.
[Here][1] is s a blog post by the Scala team about it.
Warnings have been either solved or preserved if useful,
trying to minimize the scope (keeping it at the single
expression scope if possible). In particular, all
remaining usages of the Scala Collection API compatibility
module have been removed.
Using the silencer plugin also apparently hid a few
remaining usages of compatibility libraries that were used
as part of the transition from Scala 2.12 to Scala 2.13
that are no longer needed. Removing those warnings
highlighted those.
changelog_begin
changelog_end
[1]: https://www.scala-lang.org/2021/01/12/configuring-and-suppressing-warnings.html
Adding
- `PersistentUserManagementStore` and `CachedUserManagementStore`,
- `UserManagementStorageBackendTemplate` and sql migrations,
- CLI flags: `--user-management-max-cache-size` and `--user-management-cache-expiry`;
And wiring `PersistentUserManagementStore` where before we had `InMemoryUserManagementStore`.
`--ghc-option=-Werror` turns warnings into errors, `--ghc-option=-Wwarn`
undoes exactly that so this clearly was not doing anything sensible.
changelog_begin
changelog_end
Somewhat error-prone, so please review carefully.
Reasons we need this:
- Some file types are not properly handled by the script.
- The only exclusion mechanism we currently have (`NO_AUTO_COPYRIGHT`)
is overly coarse.
CHANGELOG_BEGIN
CHANGELOG_END
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Set --enable-scenarios to False by default
changelog_begin
changelog_end
* Enable scenarios for shake test
* Add --enable-scenarios=yes for //compiler/lsp-tests
* Add enable_scenarios arg to rules_daml/daml.bzl helpers
* daml_compile
* daml_build_test
* daml_test
* Add enable_scenarios to ledger/test-common helper da_scala_dar_resources_library
* Add --enable-scenarios=yes for //ledger/test-common
* Remove unused scenario in //ledger/test-common PingPong test
* Add --enable-scenarios=yes for //daml-lf/tests
* Add --enable-scenarios=yes for //language-support/java tests
* Add --enable-scenarios=yes for group-chat example
* Add --enable-scenarios for //ledger/sandbox-perf LargeTransaction test
* Add --enable-scenarios=yes for //docs tests
* Add --enable-scenarios=yes for //daml-lf/scenario-interpreter tests
* format bazel
* Add --enable-scenario=yes for 'daml_doc_test's
* Add --enable-scenarios=yes for DamlDocTestIntegration
* Add --enable-scenarios=yes for DamlcTest
* Add --enable-scenarios=yes for Test/DataDependencies
* Add --enable-scenarios=yes for daml-ghc-deterministic.sh
* Add --enable-scenarios=yes for Test.IncrementalBuilds
* Add --enable-scenarios=yes for tests:memory-bond-trading and tests:memory-examples
* Add --enable-scenarios=yes for daml-assistant/integratio-tests
Apparently this works very poorly with damlc_legacy which fails to
find hpp after the latest nixpkgs upgrade. Somewhat confusingly it
only fails to find that on release builds and I don’t understand how
it ever worked but this seems more sensible anyway.
changelog_begin
changelog_end
* Generating random payload of configurable size in ledger-api-bench-tool command submission
CHANGELOG_BEGIN
- [Integration Kit] - ledger-api-bench-tool can generate test contracts with configurable payload size.
CHANGELOG_END
* Fix for Scala 2.12
* Multi-template support for command submission in the ledger-api-bench-tool [DPP-659] (#11365)
* Added multi-template support for command submission in the ledger-api-bench-tool
CHANGELOG_BEGIN
- [Integration Kit] - Added multi-template support for command submission in the ledger-api-bench-tool
CHANGELOG_END
* Simplified distribution calculation
* Improved throttling - set max 100 in-flight commands
* Reorder the observers for StreamConsumer to first filter and then take just one element, not the other way around
CHANGELOG_BEGIN
CHANGELOG_END
* Refactor the test to get better stacktraces during failures
* Remove unused import
* [Daml error codes API] Further implementations
* Implements ErrorCode.asGrpcError (and test)
* Error code logging now accepts correlation id and an extra context map
* Full error context is included into enriched logging context
CHANGELOG_BEGIN
CHANGELOG_END
* Fixed Scala 2.12 compilation issues
* Stop using `controller … can` syntax in ledger tests
We’re planning to deprecate that in SDK 2.0 to reduce the confusion
around the implicit observer behavior so in preparation for that, this
PR drops the syntax from all ledger tests.
changelog_begin
changelog_end
* shuffle let above usage
changelog_begin
changelog_end
Adding support for accepting server's private key as an encrypted file (since storing unencrypted private key in a file system might be a risk).
Encrypted private key is assumed to be encrypted using AES or similar algorithm. The details necessary to decrypt it are be obtained from a secrets server over HTTP as JSON document. The URL to secret's server is supplied through the new `--secrets-url` CLI parameter.
One can supply private in either plaintext (old behavior) or ciphertext: if a private key's file ends with .enc suffix it is assumed to be ciphertext. Otherwise it is assumed to be plain text.
CHANGELOG_BEGIN
- [DPP-418] [Participant] Add support for supplying server's private key as an encrypted file and then decrypting it with the help of a secrets server.
CHANGELOG_END
* [Divulgence pruning] Conformance tests implementation
CHANGELOG_BEGIN
[Integration kit] Extended the Ledger API test tool with tests for the pruning of all divulgence events.
CHANGELOG_END
* Addressed review comments
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
I don't find the output from Bazel very helpful most of the time. It
does, however, produce a lot of noise which I have to filter through
when something else goes wrong.
CHANGELOG_BEGIN
CHANGELOG_END
* A test case for crashing participant in a multi-node env with divulged contracts
* Fixed test naming
CHANGELOG_BEGIN
CHANGELOG_END
* Additional usability check for the disclosed contract
* A newline
* Verify that fetching an archived contract is not possible
* Second test case
* A `ledger-api-test-tool` test case for divulging a contract with key twice on multiple participants [DPP-433] (#9970)
* A test case for divulging a contract with a key twice on multiple participants
CHANGELOG_BEGIN
CHANGELOG_END
* Use autogenerated archive method
* Trigger build
* Increased timeout for TransactionServiceIT:TXInvisibleTransactionTreeByEventId
* Add ledger API test tool tests for rollback projections
This adds 3 tests for projections under rollback nodes.
The first one is relatively clear hopefully and tests divulgence.
The other two are a bit more intricate. For both of those we can also
not test too much via the ledger API since we don’t actually get
access to rollback nodes. However, it still seems useful to at least
exercise those code paths and make sure they don’t do anything
horribbly wrong.
The second test tests the normalization rules from
https://github.com/digital-asset/daml/blob/main/docs/source/concepts/ledger-model/ledger-exceptions.rst#privacy
The last one tests a more complex structure with deeply nested
rollback nodes and different informees.
changelog_begin
changelog_end
* Update ledger/test-common/src/main/daml/semantic/Exceptions.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Add race condition tests for exceptions
This PR addresses
https://github.com/digital-asset/daml/pull/9400#pullrequestreview-634770251
and adds tests that match RaceConditionTests but to the read side in a
rollback (we cannot do writes in rollbacks, they are rolled back :)).
The tests are as close as possible to the other race condition tests
to ease maintenance and reduce confusion.
changelog_begin
changelog_end
* remove commented lines
changelog_begin
changelog_end
* Disable accidentally enabled ClosedWorldIT
changelog_begin
changelog_end
* Add Ledger API test tool tests for exceptions
changelog_begin
changelog_end
* Update daml-lf/language/daml-lf.bzl
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Address review comments
changelog_begin
changelog_end
* Shuffle around test
changelog_begin
changelog_end
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Check visibility for by-key operation of local contracts
fixes#9454
I tried out two approaches for this:
1. The one here where we add a new callback. This has the advantage
that the engine remains oblivious to visibility checks. They are all
done outside and the engine doesn’t even know about the reading
parties.
2. Make the engine aware of the reading parties. A start of that is in
#9458.
Both work in principle but I ended up going for 1 in the end. Doing
half of the visibility checks outside the engine and half inside just
seems worse than the current state.
changelog_begin
- [Daml Engine] Fix a bug where it was possible to
fetch/lookup/exercise a local contract by key even if the reading parties
are not stakeholders. See #9454 for details.
changelog_end
* Disable new test on Canton
changelog_begin
changelog_end
* Exclude from compat tests
changelog_begin
changelog_end
* s/LocalLookup/LocalFetch/
changelog_begin
changelog_end
* Address review
changelog_begin
changelog_end
* test-common: Simplify RaceTests.daml using `isSome`.
* ledger-api-test-tool: Make IntelliJ happy about RaceConditionIT.
It doesn't seem to like the combination of `Future.traverse` and ranges.
* ledger-api-test-tool: Pull out methods and re-use when possible.
I prefer helper methods to be in the companion object where possible.
CHANGELOG_BEGIN
CHANGELOG_END
* Early draft of the race condition ITs
* Archival vs Successful lookup by key test
* More descriptive failure messages
* Unsuccessful lookup vs non-transient creation test
* Double-archival test
* Fixed a test case name
* Archival vs Creation order test
* Reduced number of test templates
* Improved race test template naming
* Helper object with transaction and template utils
* Simplified transaction util
* Fixed wrong choice name
* Removed redundant println
* Formatted code changes
* Minor change
* CHANGELOG_BEGIN
- Integration Kit - added a test suite for race condition to the ledger-api-test-tool
CHANGELOG_END
* Removed unnecessary sorting of transactions
* Added explanatory comments to test cases
* Mechanism for running ledger-api-test-tool test cases multiple times
* Running each race condition test case 5 times
* Fixed WWArchiveVsNonTransientCreate test case
* Fixed flakiness of RWArchiveVsNonConsumingChoice
* Disabled RaceConditionIT in Canton tests
* Formatted code changes
* Moved RaceConditionIT to conformance tests with unique contract keys mode on for Canton
* Nicer delay mechanism
* Improved WWArchiveVsNonTransientCreate to take contention into account
* Fixed RWTransientCreateVsNonTransientCreate conditions for Canton
* Increased the delay before reading the transaction trees stream to 1 second
* Fixed incorrect conformance tests definition for RaceConditionIT
* Running race condition tests sequentially to avoid timeouts
* Simplified race condition test case definition
* Return sum of durations for repeating test cases in the ledger-api-test-tool
* Reverted previous change with computing sum of durations
* Exclude RaceConditionIT from sandbox-on-x conformance tests
* Print the number of a test run only for cases when the number of repetitions is > 1
* Fixed RWArchiveVsFetch scenario
* Expose libraries for integration testing purposes
The motivation of these changes is to eliminate manual work and reduce duplication between the SDK and oem-integration-kit repos by reusing the same test fixture for integration testing participant state implementations. Also, the DARs required for running these tests won't need to be manually updated.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix a concurrency issue in integration tests
* Fix Bazel error
* Fix conflict resolution
* Move inline daml-lf to separate dar files
* Add a comment
* Add a missing artifact
* Extract method
* Remove maven tags
* Add a macro for Scala libraries with dar resources
* Improve the macro
* Add missing artifact
* Simplify the tests
* Format signature
* Fix the maven tag
* Add missing copyright headers
* Format bazel files
* Make //ledger/test-common lf version dependent (to avoid jar hell)
* Move da_scala_dar_resources_library to a separate bzl file
* Add missing artifacts
Co-authored-by: Hubert Slojewski <hubert.slojewski@tesco.com>
* Ledger API test tool: test against legacy and preview version.
+ use dictionary instead of alias to map version keywords to LF version
CHANGELOG_BEGIN
CHANGELOG_END
improve previous generalization from #8695
- use lf version instead keyword (like 'stable', 'latest', 'dev') to
tag actual target. This will allow two keywords to map to the same
versions without doing twice the compilation/test work.
- use alias to map keywords tag target to versioned tag target.
- move package manage dar to test_commong.
CHANGELOG_BEGIN
CHANGELOG_END
fixes#8498
This fixes the error in 2.13 wtr to the location change of Predef. It
doesn’t yet address the warning wtr to the import of higherKinds. For
now, our build ignores that warning. Trying to figure out if we can
get away with a breaking change here or if we need to hide that change
behind a flag but either way, no need to block fixing the actual error
on that.
changelog_begin
changelog_end
* Add CLI options to enable daml-lf dev version
* Compile test-common with multiple compiler versions
* Run the ledger API test tool with multiple compiler versions
changelog_begin
changelog_end
* Fix scaladoc issues
* Apply reviewer comments
regarding lf_preview_version
* --client-auth param for kvutils app
CHANGELOG_BEGIN
-- enabled --client-auth for kvutils config
CHANGELOG_END
* TLS integration tests for StandaloneApiServer client authorization parameter
* Refactored TlsFixture to reduce duplication
* Improved description of the client-auth parameter
* Added test cases with invalid certificates
* Improved readability of the test cases
* Formatted changes
* Port the rest of //ledger/... to Scala 2.13
draw the rest of the fcking owl
Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).
changelog_begin
changelog_end
* Less symbols
changelog_begin
changelog_end
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Port damlc dependencies to Scala 2.13
I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).
So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).
If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.
changelog_begin
changelog_end
* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* fixup lawlesstraversal
changelog_begin
changelog_end
* less iterator more view
changelog_begin
changelog_end
* document safety of unsafeWrapArray
changelog_begin
changelog_end
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
