* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
I don't find the output from Bazel very helpful most of the time. It
does, however, produce a lot of noise which I have to filter through
when something else goes wrong.
CHANGELOG_BEGIN
CHANGELOG_END
* A test case for crashing participant in a multi-node env with divulged contracts
* Fixed test naming
CHANGELOG_BEGIN
CHANGELOG_END
* Additional usability check for the disclosed contract
* A newline
* Verify that fetching an archived contract is not possible
* Second test case
* A `ledger-api-test-tool` test case for divulging a contract with key twice on multiple participants [DPP-433] (#9970)
* A test case for divulging a contract with a key twice on multiple participants
CHANGELOG_BEGIN
CHANGELOG_END
* Use autogenerated archive method
* Trigger build
* Increased timeout for TransactionServiceIT:TXInvisibleTransactionTreeByEventId
* Add ledger API test tool tests for rollback projections
This adds 3 tests for projections under rollback nodes.
The first one is relatively clear hopefully and tests divulgence.
The other two are a bit more intricate. For both of those we can also
not test too much via the ledger API since we don’t actually get
access to rollback nodes. However, it still seems useful to at least
exercise those code paths and make sure they don’t do anything
horribbly wrong.
The second test tests the normalization rules from
https://github.com/digital-asset/daml/blob/main/docs/source/concepts/ledger-model/ledger-exceptions.rst#privacy
The last one tests a more complex structure with deeply nested
rollback nodes and different informees.
changelog_begin
changelog_end
* Update ledger/test-common/src/main/daml/semantic/Exceptions.daml
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Add race condition tests for exceptions
This PR addresses
https://github.com/digital-asset/daml/pull/9400#pullrequestreview-634770251
and adds tests that match RaceConditionTests but to the read side in a
rollback (we cannot do writes in rollbacks, they are rolled back :)).
The tests are as close as possible to the other race condition tests
to ease maintenance and reduce confusion.
changelog_begin
changelog_end
* remove commented lines
changelog_begin
changelog_end
* Disable accidentally enabled ClosedWorldIT
changelog_begin
changelog_end
* Add Ledger API test tool tests for exceptions
changelog_begin
changelog_end
* Update daml-lf/language/daml-lf.bzl
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Address review comments
changelog_begin
changelog_end
* Shuffle around test
changelog_begin
changelog_end
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Check visibility for by-key operation of local contracts
fixes#9454
I tried out two approaches for this:
1. The one here where we add a new callback. This has the advantage
that the engine remains oblivious to visibility checks. They are all
done outside and the engine doesn’t even know about the reading
parties.
2. Make the engine aware of the reading parties. A start of that is in
#9458.
Both work in principle but I ended up going for 1 in the end. Doing
half of the visibility checks outside the engine and half inside just
seems worse than the current state.
changelog_begin
- [Daml Engine] Fix a bug where it was possible to
fetch/lookup/exercise a local contract by key even if the reading parties
are not stakeholders. See #9454 for details.
changelog_end
* Disable new test on Canton
changelog_begin
changelog_end
* Exclude from compat tests
changelog_begin
changelog_end
* s/LocalLookup/LocalFetch/
changelog_begin
changelog_end
* Address review
changelog_begin
changelog_end
* test-common: Simplify RaceTests.daml using `isSome`.
* ledger-api-test-tool: Make IntelliJ happy about RaceConditionIT.
It doesn't seem to like the combination of `Future.traverse` and ranges.
* ledger-api-test-tool: Pull out methods and re-use when possible.
I prefer helper methods to be in the companion object where possible.
CHANGELOG_BEGIN
CHANGELOG_END
* Early draft of the race condition ITs
* Archival vs Successful lookup by key test
* More descriptive failure messages
* Unsuccessful lookup vs non-transient creation test
* Double-archival test
* Fixed a test case name
* Archival vs Creation order test
* Reduced number of test templates
* Improved race test template naming
* Helper object with transaction and template utils
* Simplified transaction util
* Fixed wrong choice name
* Removed redundant println
* Formatted code changes
* Minor change
* CHANGELOG_BEGIN
- Integration Kit - added a test suite for race condition to the ledger-api-test-tool
CHANGELOG_END
* Removed unnecessary sorting of transactions
* Added explanatory comments to test cases
* Mechanism for running ledger-api-test-tool test cases multiple times
* Running each race condition test case 5 times
* Fixed WWArchiveVsNonTransientCreate test case
* Fixed flakiness of RWArchiveVsNonConsumingChoice
* Disabled RaceConditionIT in Canton tests
* Formatted code changes
* Moved RaceConditionIT to conformance tests with unique contract keys mode on for Canton
* Nicer delay mechanism
* Improved WWArchiveVsNonTransientCreate to take contention into account
* Fixed RWTransientCreateVsNonTransientCreate conditions for Canton
* Increased the delay before reading the transaction trees stream to 1 second
* Fixed incorrect conformance tests definition for RaceConditionIT
* Running race condition tests sequentially to avoid timeouts
* Simplified race condition test case definition
* Return sum of durations for repeating test cases in the ledger-api-test-tool
* Reverted previous change with computing sum of durations
* Exclude RaceConditionIT from sandbox-on-x conformance tests
* Print the number of a test run only for cases when the number of repetitions is > 1
* Fixed RWArchiveVsFetch scenario
* Expose libraries for integration testing purposes
The motivation of these changes is to eliminate manual work and reduce duplication between the SDK and oem-integration-kit repos by reusing the same test fixture for integration testing participant state implementations. Also, the DARs required for running these tests won't need to be manually updated.
CHANGELOG_BEGIN
CHANGELOG_END
* Fix a concurrency issue in integration tests
* Fix Bazel error
* Fix conflict resolution
* Move inline daml-lf to separate dar files
* Add a comment
* Add a missing artifact
* Extract method
* Remove maven tags
* Add a macro for Scala libraries with dar resources
* Improve the macro
* Add missing artifact
* Simplify the tests
* Format signature
* Fix the maven tag
* Add missing copyright headers
* Format bazel files
* Make //ledger/test-common lf version dependent (to avoid jar hell)
* Move da_scala_dar_resources_library to a separate bzl file
* Add missing artifacts
Co-authored-by: Hubert Slojewski <hubert.slojewski@tesco.com>
* Ledger API test tool: test against legacy and preview version.
+ use dictionary instead of alias to map version keywords to LF version
CHANGELOG_BEGIN
CHANGELOG_END
improve previous generalization from #8695
- use lf version instead keyword (like 'stable', 'latest', 'dev') to
tag actual target. This will allow two keywords to map to the same
versions without doing twice the compilation/test work.
- use alias to map keywords tag target to versioned tag target.
- move package manage dar to test_commong.
CHANGELOG_BEGIN
CHANGELOG_END
fixes#8498
This fixes the error in 2.13 wtr to the location change of Predef. It
doesn’t yet address the warning wtr to the import of higherKinds. For
now, our build ignores that warning. Trying to figure out if we can
get away with a breaking change here or if we need to hide that change
behind a flag but either way, no need to block fixing the actual error
on that.
changelog_begin
changelog_end
* Add CLI options to enable daml-lf dev version
* Compile test-common with multiple compiler versions
* Run the ledger API test tool with multiple compiler versions
changelog_begin
changelog_end
* Fix scaladoc issues
* Apply reviewer comments
regarding lf_preview_version
* --client-auth param for kvutils app
CHANGELOG_BEGIN
-- enabled --client-auth for kvutils config
CHANGELOG_END
* TLS integration tests for StandaloneApiServer client authorization parameter
* Refactored TlsFixture to reduce duplication
* Improved description of the client-auth parameter
* Added test cases with invalid certificates
* Improved readability of the test cases
* Formatted changes
* Port the rest of //ledger/... to Scala 2.13
draw the rest of the fcking owl
Omitted for now are the ledger API test tool which has a dependency
only compatible with 2.12 and the generated code of the Scala
codegen (the codegen compiles and runs with 2.13, the generated code
does not).
changelog_begin
changelog_end
* Less symbols
changelog_begin
changelog_end
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
* Port damlc dependencies to Scala 2.13
I got a bit fed up by the fact that going directory by directory
didn’t really work since there are two many interdependencies in
tests (e.g., client tests depend on sandbox, sandbox tests depend on
clients, engine tests depend on DARs which depend on damlc, …).
So before attempting to continue with the per-directory process, this
is a bruteforce approach to break a lot of those cycles by porting all
dependencies of damlc which includes client bindings (for DAML Script)
and Sandbox Classic (also for DAML Script).
If this is too annoying to review let me know and I’ll try to split it
up into a few chunks.
changelog_begin
changelog_end
* Update daml-lf/data/src/main/2.13/com/daml/lf/data/LawlessTraversals.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* fixup lawlesstraversal
changelog_begin
changelog_end
* less iterator more view
changelog_begin
changelog_end
* document safety of unsafeWrapArray
changelog_begin
changelog_end
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
This commit fixes a few copyright headers that have been missed in the
automatic update on Jan 1, as well as the generation code in the compat
workspace so it generates the right headers.
CHANGELOG_BEGIN
CHANGELOG_END
* Remove single-party check
CHANGELOG_BEGIN
- [Ledger API] The ledger API now supports multi-party submissions.
In order to use multi-party submissions, use the new act_as and
read_as fields in submission requests.
CHANGELOG_END
* Remove usage of temporary SubmitterInfo methods
* Fix validator tests
* Fix auth test
* Add multi-party tests to the ledger API
* Fix compile errors
* Improve tests
* Remove temporary single-party method from SubmitterInfo
* Remove temporary single-party method from SubmitterInfo companion object
* Remove temporary single-party method from TxEntry
* Run multi-party submission ITs for ledger-on-memory and ledger-on-sql
* Minor improvement
Co-authored-by: Kamil Bozek <kamil.bozek@digitalasset.com>
* ledger-api: Use `proto_jars`.
CHANGELOG_BEGIN
- [Ledger API] The Scala JARs containing the gRPC definitions no longer
contain the *.proto files used to generate the ScalaPB-based classes.
CHANGELOG_END
* Create a source JAR for *.proto files in `proto_jars`.
* ledger-api: Publish the protobuf sources as "ledger-api-proto".
CHANGELOG_BEGIN
- [Ledger API] The *.proto files containing the gRPC definitions are now
provided by a new Maven Central artifact, with the group "com.daml"
and the artifact name "ledger-api-proto".
CHANGELOG_END
* release: We don't need the "main-jar" option.
* Bazel: Proto JARs will always have a Maven artifact suffix.
* Bazel: Simplify Protobuf source file TAR and JAR targets.
* Bazel: Extract out Protobuf functions.
Adds TLS certificates revocation checking in the LedgerApiServer using the OCSP.
The feature is implemented by setting global JVM properties.
Integration tests for the new feature include spinning up a local OCSP responder using the openssl command.
CHANGELOG_BEGIN
- new CLI option --cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer
- documentation about the new feature
CHANGELOG_END
* test-common: Make FiniteStreamObserver more lenient.
We can't protect against all possible race conditions; we just have to
accept that sometimes `onNext` will be called after `onCompleted`, or
`onCompleted` will be called twice, or…
CHANGELOG_BEGIN
CHANGELOG_END
* test-common: Avoid calling `onCompleted` twice in `TimeBoundObserver`.
* Suppress debug and trace logging for Sandbox tests.
* participant-integration-api: Don't submit config after shutting down.
* sandbox: When testing the reset service, wait for the ledger config.
If we don't wait for the ledger configuration, we could potentially ask
for completions before the server is fully ready.
* sandbox-common: Reduce the number of reset attempts in testing.
Now we're also checking for ledger configuration, the time for a reset
to be "accepted" is longer, so 5 tries in 30 seconds is a bit flaky.
Reducing to 4 seems reasonable.
* sandbox: Stop marking the reset service tests as flaky.
I don't have conclusive proof that they're no longer flaky, but I'm
fairly confident. We can revert this commit if they turn out to still
need some attention.
CHANGELOG_BEGIN
CHANGELOG_END
* sandbox-common: Check the configuration shows up in reset service tests.
* sandbox-common: Factor out logback-test.xml.
* concurrent: Tag DirectExecutionContext.
1. Tag `DirectExecutionContext` as `ExecutionContext[Nothing]`, thereby
stating that it works for any tagged `Future`.
2. Move `DirectExecutionContext` to the _libs-scala/concurrent_
library, as it requires it and it's tiny.
CHANGELOG_BEGIN
CHANGELOG_END
* concurrent: Fix the privacy of `DirectExecutionContextInternal`.
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Make the contract keys test fail if the transaction is not rejected by the committer
- First, this removes the assert statement in the DAML model, so that the
transaction validation doesn't fail with a transaction abort.
- Second, the gRPC error message should contain Disputed. `InvalidLookup`
should only be used in Sandbox Classic, but due to code sharing, the
validation that triggers this error is also run for Sadbonx.
* Fix the key lookup in kvutils
The current way of transaction validation and key lookup does not work
in the following scenario:
- the transaction coming from the participant has a negative key lookup
- the contract key state is loaded, which points to a contract that has
been created by another transaction since the command interpretation
on the participant
- the contract state for the contractId inside the contract key state
has not been loaded, therefore the visibility and activeness check in
`ProcessTransactionSubmission.lookupKey` returns `None`, even though
there is in fact a contract with such a key.
To mitigate this issue, the `lookupKey` function must not check
activeness or visibility. If the submitter is not allowed to load the
contract via a key lookup, the authorization check should catch that.
Any other situation that is inconsistent should result in a rejection of
the transaction.
A small side effect of this change is that the `lookupContract` function
might try to load a contract that wasn't loaded from the kv state
earlier. This only happens because we first rebuild the entire
transaction before we compare it to the original transaction.
Previously we threw an exception in this case, but this results in a rather
ugly abort of the reinterpretation of the transaction. Therefore the
`lookupContract` function just returns `None`. Eventually the
transaction will be rejected during validation of the nodes, because
the `LookupByKey` nodes will be different.
This would not happen if we reinterpreted the transaction lazily and
compared the nodes as soon as they are produced with the corresponding
node of the original transaction.
CHANGELOG_BEGIN
[kvutils] Fixed the validation of key lookups.
See issue `#5562 <https://github.com/digital-asset/daml/issues/5562>`__.
[Participant Server, Ledger API Test Tool] Use rejection reasons of the participant-state API instead of sandbox specific ones.
CHANGELOG_END
* Validate causal monotonicity of used contract keys
When using LookupByKey, the return contract isn't necessarily fetched or used
any other way. This means that we need an extra pass over all contract keys to
ensure that causal monotonicity is respected. To be able to do this,
there is now a new field `DamlContractKeyState.active_at`, which is set to the
same value `DamlTransactionEntry.ledger_effective_time`.
An additional check in `ProcessTransactionSubmission` uses this field to verify
causal monotonicity.
* Add failing test
The test can produce false negatives,
but locally it fails 10 out of 10 times.
* Stop deduplicating commands after rejections
Fixes#5338.
CHANGELOG_BEGIN
CHANGELOG_END
* Only deduplicate successful transactions
* Use pass instead of pure unit
* Handle exceptions
* Draft of PingPong throughput and latency benchmarks for on-mem and on-sql ledgers
* Augment `ParticipantTestContext` and remove `LedgerApiServer` hack
* Separate performance tests into distinct category with concurrency 1
* 🎨
* Package performance tests in separate DAR
* Have performance tests excluded by default and run exclusively if passed
* Fix rebase
* Simplify `BenchmarkReporter`
* Make `concurrencyOverride` into an `Option`
* Clarify command line usage, prevent regular and perf. tests together
* Fix preventing regular and perf. tests together
* Split `PingPong`, `PingPongExplode` and `Cycle` benchmarks' model
CHANGELOG_BEGIN
- [TestTool] Add `PingPong` performance envelope test
CHANGELOG_END
* Explicitly name `concurrencyOverride`
* Fix formatting
* Lower bar for CI run of performance envelope tests
* Make benchmark output file configurable
* Improve messages and report config option name
* Use exit status 64 for "bad command line usage" as in BSD
Packages com.digitalasset.daml and com.daml have been unified under com.daml
Ledger API and DAML-LF DEV protos have also been moved from `com/digitalasset`
to `com/daml` on the file system.
Protos for already released DAML LF versions (1.6, 1.7, 1.8) stay in the
package `com.digitalasset`.
CHANGELOG_BEGIN
[SDK] All Java and Scala packages starting with
``com.digitalasset.daml`` and ``com.digitalasset`` are now consolidated
under ``com.daml``. Simply changing imports should be enough to
migrate your code.
CHANGELOG_END
Avoiding `damlc compile/package` commands (which we would like to deprecate), and replace with plain `damlc build` together with a post dar->dalf extraction step in the couple of places where we actually want the .dalf for testing.
changelog_begin
changelog_end
* Share test certificates
This is primarily an attempt at making sure my contribution stats
remain negative but I think it’s a nice cleanup. The only difference
in the certs used by daml-helper which are now used everywhere is that
they use a different CN for the CA and the server. This is required to
make openssl happy (which is used by the daml-helper).
changelog_begin
changelog_end
* Fix script and trigger tests
