* Initial version for configuration submissions and authorization
* Refactor config submission based on review. Add tests.
* Cleanup test output and comments
* kvutils test-suite
- Add unit tests for kvutils
- Thread the input state into Commit monad
- Fix a bug in handling rejected transactions (from previous refactoring)
* Address PR review
- Add submissionId to configuration changes.
- Revert the dedup of the ParticipantNotAuthorized message. This did not buy much
and we need to duplicate it anyway when modularizing kvutils.
This test represents the behavior of contract keys before the change of behavior in DAML LF 1.DEV.
The existing ContractKeys test is renamed to ContractKeysSubmitterIsMaintainer.
The `LedgerTestSuiteRunner` now uses a fixed number of
threads for the runner threadpool. This way we don't flood
the CPU with threads when using a CachedThreadPool.
* New acceptance test tool prototype
* Fix transaction filter helper
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308081386
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r308082432
Moreover, creates specific helpers for test DAML code.
* Move helpers from session to context
Remove a circular dependency between session and context, and limits the
usage of implicits.
* Run tests with ledgers out of process
* Prepare for drop in
* Temporary timeout increase
* Move bulk of the logic to the test runner
* Have a map of tests for CLI integration
* Timeout per-test, runner uses context, renaming
* Allow usage of time service, integrate semantic tests
* Integrate CLI SSL configuration
* Integrate failure expectation CLI option
* Integrate with failure output verbosity CLI option
* Integrate with test listing and picking CLI options
* Integrate with timeout scaling CLI option
* Integrate with command TTL CLI option
* Review CODEOWNERS
* Drop previously unimplemented CLI parameters
* Expand test todo list to encompass latest additions
* Deduplicate some testing infrastructure, swap in new test tool
* Remove unused rule
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311045444
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311051974
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311035287
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311055290
* WIP
* Make gRPC thread factory spawn non-deamon threads
* Remove debug prints, complete divulgence test (missing assertion messages)
* Try to detect assertion failure line number
* Add Sandbox binary that starts an ephemeral postgres instance
This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.
Fixes#1543
* Add Sandbox binary that starts an ephemeral postgres instance
This allows us to use the client_server_test bazel macro to run against
the sandbox backed by postgres.
Fixes#1543
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r311400695
* Remove TlsConfiguration duplicate, move original to commons
* Make assertion more readable
Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com>
* Fix build errors introduced in previous commit
- exclude time service tests from reference-v2
- add copyright header to TlsConfiguration
* Complete divulgence test and address review comments
Restores comments from original divulgence tests and strives to be as
close to them as possible.
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311429169
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431156
- Address https://github.com/digital-asset/daml/pull/2282#discussion_r311431913
* Delete old divulgence tests
* Restore methods to fetch contracts by template identifier
* Run conformance tests on in-memory sandbox
* Run conformance tests on the persistent sandbox
* Remove workflow identifier workaround
* New contract keys test (#2452)
* Set channel type for LedgerSession
* Add ContractKeysIT
* Run tests against reference-server v2
* Removed the old ContractKeysIT. However, ...
... I cannot remove ContractKeysChecks completely, because we have
another test path for DARs created prior to DAML-LF 1.6, which doesn't
have the submitter == maintainer restriction for contract keys.
* Use codegen instead of custom code
* Add missing copyright headers
* Drop sandbox conformance test suite (seems to not work on Windows CI)
* Make time simpler
* Test tool tests to run only semantic tests
* Make CI behave more consistently
* Add missing files to run tests on Postrgres
* Fix flaky divulgence test, introduce readable identifiers
* Increase sandbox conformance tests timeout
* Address https://github.com/digital-asset/daml/pull/2282#discussion_r312363246
* Reintroduce test resource extraction
* Read semantic tests from the classpath rather than Bazel
* Use codegen types for ContractKeys integration test
allocateParty returns Primitive.Party and create and exercise also
accept Primitive.Party. This way we don't have to keep a String and
Primitive.Party representation around.
* Bump timeout for sandbox conformance tests
* Remove dependency on Bazel runfiles
It looks like that closing the LedgerApiServer before closing the
AkkaExecutionSequencerPool (AESP) leads to a non-deterministic bunch of dead
letter warnings when stopping sandbox.
Closing the AESP first avoids these warnings.
Fixes#1886
This happened because we translate empty workflow ids to None in the
API-to-domain translation layer. When we store the workflow id, we actually
convert the None case back to an empty string (and not a null value).
When loading the workflow id from the database, we parse the column as a regular
LedgerString, but unfortunately these cannot be empty. Therefore using empty workflow
ids on postgres has been broken since workflow id has changed from a tagged string
to LedgerString.
* Read stakeholders through the active contract service
Fixes#2070
Also standardizes the use of American English spelling for `serialize`
and `deserialize`. Standardize. Serialize. Americanize.
* Remove Haskell bindings test workaround
* Address https://github.com/digital-asset/daml/pull/2316#discussion_r308193661
This is a breaking change on the ledger api. So far we could rely on the
assumption that contractId == eventId. This assumption doesn't hold
anymore in a daml-on-x setting, where the eventId is created by the
api server, but the absolute contractId is created by the ledger
implementation.
Instead of going through weird contortions to store the relevant data in
the existing database schema, it is more viable in the long term to
remove that field and instead provide facilities to lookup transactions
by contractId.
Contributes to #2068.
* keeping loaded packages after reset
removing unnecessary extra reference
changelog
making InMemoryPackageStore immutable
* using Instant.now when loading packages in IndexServer
* loading Dars from ZipInputStreams
formatting and cleanup
fixing DarReaderTest not compiling
fmt
using IOUtils to read the ZipInputStream
fixing CodeGen
fixing broken legacy DAR loading
fixed v2 ReferenceServer
cleanup
* fixing rebase conflicts
* introduced a fromFile method to avoid having to do boiler-plating where the input is a File
* addressing leaking resources
* cleanup
* inlining closures
* closing the darStream
* make every PublicPackageUploaded update contain exactly one package
* implement update subscriptions that start with offsets with the same major number
* address review comments
* Do not convert the transaction id to Long in LedgerBackedIndexService
The InMemoryLedger does this, because it can make certain assumptions
(like transactionId == offset), but the sql based ledger cannot do that,
because the transactionId may be produced by another backend and thus
have a different format.
* Move tests that use contract keys into their own integration test file
This allows us to run the other tests via the ledger api test tool with
the postgres indexer while InMemoryKVParticipantState doesn't support
contract keys yet.
* Remove a few occurrences of LedgerBegin from TransactionServiceIT.
* Rename CommandTransactionChecksNo1866IT to ContractKeysNo1866IT
* Enable TransactionServiceTests,DivulgenceIT,TransactionBackpressureIT for postgres index|er
* format all the things
* Address a few review comments.
See https://github.com/digital-asset/daml/pull/2032#pullrequestreview-258771927
* Extract reusable base class for ContractKeys tests
This is needed because Bazel's scala_test_suite isolates the tests from
another, and one test cannot extend another one.
* Classify the reference-v2 conformance tests with a long timeout
* connenct LedgerAPI party and package management with KVUtils
* formatting
* address review comments
* add participantId parameter to sandbox and indexer
* annotate TODO's with github issue numbers
* check that submitter is in maintainers when looking up keys
Fixes#1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.
* add UNTIL-LF to run tests up to a certain version of DAML-LF
* name targets for DAML tests better
* add notes about DAML-LF changes
* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly
* add scenario tests for #1866
* add warnings about future key behavior in docs
* use flag rather than version when executing
* Add external offset column to parameters
We only ever need to use the external offset when restarting the api
server from a persistent data store to resume the ReadService state
update stream from the last processed offset.
In the PostgresIndexer we then need to store the external offset while
adding a new ledger entry.
* Allow disabling the jmx reporter in metricsmanager
* Implement PostgresIndexer component
* Introduce read-only Ledger and Dao
* SandboxEventIdFormatter treats transactionId as opaque string
* Introduce LedgerBackedIndexService
This can be reused both in the Sandbox and the ApiServer
* Implement PostgresIndex component
* Reference Service based on kvutils-v2 and postgres(index|indexer)
* Disable conformance test for ReferenceServer
Adds a PostgreSQL package store to the sandbox.
Previously, packages were only stored in memory and
specified as CLI arguments. So if a user uploaded a
package at run time, he needed to add the package to
the CLI when restarting the sandbox.
* adapting docs to match state as is regarding not supporting dalf files
* patching release notes instead
* validating for ZIP files in Cli parser
* guarding for parsing non zipped archives
* Update release-notes.rst
* Update unreleased.rst
Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>
* Update unreleased.rst
* Update release-notes.rst
* Update release-notes.rst
Moves party allocation to its own WriteService trait (similar to package upload).
Changes uploadDar to uploadPackages, so that the write service does not depend on the DAR file format.
Adds or improves many comments.
* Introduce contract key to exercise node
Track the contract key in NodeExercises. This is in
preparation for adding the key into exercise events, and
for support for contract keys in kvutils, where we need
to be able to compute the outputs of the submission, e.g.
the contract key that would be unset by a consuming exercise.
This introduces transaction version 8.
* Move release note on contract keys in exercises to unreleased.rst
* Address Stephen's review
- Drop changelog entry for this change as it is not user facing
- Add info on version 8 to transaction.rst. Fix typo in version 7.
- Reorder VersionTimeline as per review
- Only compare the NodeExercise 'key' if it is set in the original
transaction (in order to compare transactions in a backwards
compatible manner)
* Unbork unreleased.rst
* Run integration tests for sandbox through the IT test suite
We currently don't run sandbox on postgres via ledger-api-test-tool. We
need to first figure out how to best do that before we can switch
sandbox to the test tool again.
* CommandClient#submitSingleCommand also uses commandUpdater
* The test reporter for the test tool now handles ignored tests
* More optional tests for ledger-api-test-tool
- TransactionBackpressureIT
- DivulgenceIT
- CommandTransactionChecksHighLevelIT
- CommandTransactionChecksLowLevelIT
Contributes to #1374.
* Add missing domain.RejectionReason cases
* Use java.time.Instant instead of damlf Timestamp
* Make applicationId, submitterParty, commandId optional in LedgerEntry.Transaction
* store archive size in `DarReader`
* rename `SandboxTemplateStore` to `SandboxPackageStore`
* store package info in `SandboxPackageStore`
* introduce package upload / read to write / index services
not tested yet, just a ton of plumbing
* WIP test the package service
* Fix build errors after rebase
* Move packages service to v2
* Ledger API client uses ledger API types
* Fix ReflectionIT
* Correctly handle uploading invalid dar files
* Fix reading DAR entry file sizes
* Improve package management IT
* Improve handling of duplicate packages
* Fix language-support build
* Use unique party and command names
* Rename lfpackage to language
* Rename SandboxPackageStore to InMemoryPackageStore
* Remove getCurrentTime ledger method
* Improve package management IT
* Move InMemoryActiveContracts and InMemoryPackageStore
* Use case object for UploadDarResult.Ok
* Address review comments
* Update release notes
Fixes#1311
* Use BazelRunfiles in test
This fixes two issues with the sandbox database migration:
* A typo in the migration script
* Error deserializing transactions due to protobuf recursion limits
Fixes#1682
