* Extract common code for command dedup conformance tests
CHANGELOG_BEGIN
CHANGELOG_END
* Revert changes for parallel tests
* Fix compile warnings
* Disable participant deduplication for KV
CHANGELOG_BEGIN
CHANGELOG_END
* Always call submitRequestAndAssertDeduplication with parties
* Fix party in CommandDeduplicationBase
* Don't test KV-based command deduplication with static time as it's broken
* Don't disable participant-side command deduplication for sandbox-next on static time
* Revert "Don't test KV-based command deduplication with static time as it's broken"
This reverts commit 3dac41aa
* Add missing dependency
* Don't cross-test this ledger API test tool with existing versions
* Add the current PR to the list of PRs that resulted in exclusions
* Apply @garyverhaegen-da's suggestion for compatibility exclusions
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
* Port file was not written to... where?
Adds the information on where the port file could not be written.
From the current phrasing it appears the intention was to add the
information but ultimately wasn't.
changelog_begin
changelog_end
* Address https://github.com/digital-asset/daml/pull/11108#pullrequestreview-770115835
As I was adding support to a sample app for the third time, I decided I
might as well clean it up a bit and add support for everyone.
This is completely optional, activated by (build-time) environment
variables, and should be trivial to remove for users who don't care
about it. In an ideal world we'd have options to `daml new` and this
would be one, but I think adding it out-of-the-box is a better
alternative than letting people figure this out on their own.
CHANGELOG_BEGIN
- The `create-daml-app` template now includes support for a third
authentication scheme (in addition to the existing "dev mode" and Daml
Hub support): Auth0.
CHANGELOG_END
We need to differentiate between 3 cases:
1. Ledger API test tool versions that only support exclusion of whole
suites.
2. Ledger API test tool versions where the tests had different names.
3. and the current state
changelog_begin
changelog_end
* Use the token from incoming requests to update the package list
changelog_begin
changelog_end
* Lazily initialize the ledger client
* Fix ee integration tests
* Fix package reloading behaviour by using a semaphore to check for ongoing updates
* Refactor out the semaphore code into a concurrency utility class
* Use correct locking for the updateTask so every thread always uses an up to date task
* Remove unused imports in utils.Concurrent & remove packages from the tests
* Hide & mark the token file cli option deprecated because we dont need it anymore and only keep it so client deployment code doesn't break
* Fix scala 2.12 build by adding more type annotations
* Update ledger-service/http-json-cli/src/main/scala/com/daml/http/OptionParser.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Readd pgkManagementClient after it was removed accidentally (but now it's lazy)
* Remove concurrent object & use atomic boolean instead of a mutex because it makes more sense
* Replace semaphore with countdownlatch
* Refactor the caching into a separate class
* Use Instant instead of LocalDateTime
* Remove that ** of bad synchonization and do stupid simple synchronization because it JUST WORKS, besides adapt when we want to reload
* Remove await in tests because it can result in buggy tests
* remove unused code in WebSocketService.scala
* Unhide the access-token-file option as per request of Stefano
* Less implicit jwts per request of Stefano
* Try making some code more readable as by request of Akshay
* Use more shark because it expresses better than flatMaps if I don't need the arg
* Move defs in predicate in WebsocketService.scala around
* Try to minimize diff further in WebsocketService.scala
* Fix build and minimize diff in WebSocketService.scala further
* Minimize diff of function getTransactionSourceForParty in WebSocketService.scala
* Share the ec in WebSocketService.scala to minimize the diff
* Minimize in function predicate in WebSocketService.scala
* Further minimize in function predicate in WebSocketService.scala
* Change some case classes to be normal classes but with apply method
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/PackageService.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Get rid of implicit jwt tokens, the world is already confusing and full of implicits enough
* Improve readability
* Integrate the new LedgerClient which does not depend on a leder id
* Fix tests
* Apply suggestions from code review
thanks to @S11001001
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Apply further review comments
* Remove outcommented code
* Deprecate access token file option in the description too
changelog_begin
- [JSON API] The cli option `--access-token-file` is now deprecated. It
is not needed anymore and you can safely remove it. Reason is that
the operations which prior required a token at startup are now done
on demand using the token of the incoming request.
changelog_end
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Add deduplication and submission id to the proto submitter info so we can build the v2 completion info
* Use v2 rejections and store the enriched model in protobuf
* Update v1 SubmissionResult to v2 SubmissionResult
* Update v1 imports to v2 imports
* Enable API deduplication for ledgers that don't have it implemented yet (all kv)
* Update tests for new rejections
* Disable buf check for oneof
* Code review - fix formatting and test assertions
* Update imports for proto changes
* Add tests for rejection conversions
* Use entry id as submission id when submission id is missing
* Calculate deduplication period for rejection based on the set deduplicate until by calculating the duration relative to record time
* Code review consistent naming and comments
* Submission id is optional in the completion info
the submissionId is not set for participant.state.v1 entries, therefore for the transition from v1 to the v2 API we need to account for the missing submission ids by making it optional in the completion info.
* Update messages and grpc status codes for rejections to keep backward compatibility
* Deprecate proto deduplicate_until
* Set deduplication value in the commit context for all the deduplication periods. This change allows us to re-enable the committer deduplication tests
* Set ledger configuration for tests
* Command dedup: In the committer set the deduplicate until context value based on the given deduplication period [KVL-1049] (#10743)
CHANGELOG_BEGIN
participant-state:
- Migrate to use only v2 Read/Write Services. This includes the use of new models for rejections/updates/submission results.
- Calculate deduplicate until for committer side deduplication based on the submitter given deduplication period
- if using the deprecated deduplicateUntil then just set the given timestamp
- if using duration then calculate the new deduplicateUntil by using the formula (submissionTime + deduplicationDuration + minSkew)
- if using offset deduplication then calculate deduplicateUntil by using the formula (submissionTime + maxDeduplicationDuration + minSkew)
- if the deduplication period is not set then we don't set deduplicateUntil
- Emit completions with status `ALREADY_EXISTS` for duplicate commands
CHANGELOG_END
* CHANGELOG_BEGIN
Fix crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participants.
Extend ledger-api-test-tool to cover the duplicate submissionId cases.
CHANGELOG_END
* Addressing review comments and fixing failing tests
* format
* address more review comments
* adopt after rebase to main
They keep causing timeouts on MacOS and nobody seems to consider them
useful enough to investigate. For now I only disabled the postgres
tests because I haven’t seen the others time out but if they do I’ll
add them as well
changelog_begin
changelog_end
changelog_begin
changelog_end
- DAML -> daml
- Assistant -> assistant
The capitalization of the latter fixed one occurrence of a capitalized 'A'
which was inconsistent with the vast majority of uncapitalized 'a's.
This ends up transitively bumping yargs-parser which has a
vulnerability but more generally I also just like being on the latest version.
changelog_begin
changelog_end
* Added a new test suite for testing limit API values - ValueLimitsIT
* Change index on participant_command_completions table
This change fixes issues with commands with large submitters number on sandbox-classic
* Added key_hash column to the ledger *state table
The new column is now the primary key of the table. Its values are hashed values of the 'key' column which allows to mitigate limitations on the index row size
* Backfill key_hash for ledger_state table
* Dynamic state table prefix form the backfill migration
* Removed redundant comments
* Backfill migration for all the db types
* Added missing copyright comment
* Fixed migration order after a rebase
* Added missing checksums for sql migrations
* Temporarily removed copyrights from one of sql migrations
* Removed unnecessary NOT NULL constraint
* Removed submitters from the index participant_command_completion_offset_application_idx in the append-only schema
* Disabled the test for old platforms
* CHANGELOG_BEGIN
- [Integration Kit] - a new test suite ValueLimitsIT for testing edge case values
- [Integration Kit] - modified index on the participant_command_completions table to avoid issuess with a large number of submitters
- [Sandbox] - added the key_hash column to the *state table
CHANGELOG_END
* Disabled concurrent testing for the ValueLimitsIT:VLLargeSubmittersNumberCreateContract test case
* Recomputed migrations checksums for the participant-integration-api
* Increase the sandbox-on-x queue size to 500
The motivation is that running conformance tests failed with RESOURCE_EXHAUSTED error due to 200 limit on the queue.
* Minor improvement
* Inlined key hashing for migrations to avoid external dependencies
* Minor improvement
* Fixed migrations hash
* Check visibility for by-key operation of local contracts
fixes#9454
I tried out two approaches for this:
1. The one here where we add a new callback. This has the advantage
that the engine remains oblivious to visibility checks. They are all
done outside and the engine doesn’t even know about the reading
parties.
2. Make the engine aware of the reading parties. A start of that is in
#9458.
Both work in principle but I ended up going for 1 in the end. Doing
half of the visibility checks outside the engine and half inside just
seems worse than the current state.
changelog_begin
- [Daml Engine] Fix a bug where it was possible to
fetch/lookup/exercise a local contract by key even if the reading parties
are not stakeholders. See #9454 for details.
changelog_end
* Disable new test on Canton
changelog_begin
changelog_end
* Exclude from compat tests
changelog_begin
changelog_end
* s/LocalLookup/LocalFetch/
changelog_begin
changelog_end
* Address review
changelog_begin
changelog_end
This fixes the exclusions added in #9218. There are two issues:
1. Old ledger API test tool versions only support excluding full test
suites.
2. Apparently SDK 1.6.0 renamend SemanticDoubleSpend to SemanticDoubleSpendBasic.
changelog_begin
changelog_end
* Fix gRPC status codes for inconsistency rejections and DamlLf errors
Also, add unit tests and exclude failing compatibility and conformance tests
CHANGELOG_BEGIN
- [Integration Kit] Fix gRPC status codes for inconsistency rejections and DamlLf errors (ContractNotFound, ReplayMismatch) by changing them from INVALID_ARGUMENT to ABORTED
CHANGELOG_END
* Fix flakiness in create-daml-app tests
waitForSelector doesn’t do the trick here since one selector is enough
to make this return. We need to wait until we actually have the
expected number. This is what waitForFollowers already did.
changelog_begin
changelog_end
* maybe I should test my code but also I don’t want to
changelog_begin
changelog_end
* fine I tested it
changelog_begin
changelog_end
* Temporarily cut compatibility tests with incompatible versions of Daml-LF
This adds more tests to the exclusions added as part of #8957
All SDK/platform compatibility tests are not filtered to use compatible
Daml-LF versions, not just those involving the Ledger API Test Tool.
changelog_begin
changelog_end
* Fix Bazel code
These cause spurious failures on CI.
As described in the code, the proper fix is to use the correct Daml-LF version for every pair.
changelog_begin
changelog_end
CHANGELOG_BEGIN
- [LF] Release LF 1.12. This version reduce the size of transaction
- [Compiler]: Change the default LF output from 1.8 to 1.11.
CHANGELOG_END
* Port readPortfile changes to compat workspace
Unfortunately we cannot easily share code between the two workspaces
so there is some stuff that is copied over and that we need to keep in
sync. This PR ports the changes from #8669 to the compat workspace.
changelog_begin
changelog_end
* maybe I should test if my code compiles but also I don’t want to
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
a9793651b3
added a new snapshot that does not include the work done as part of
793fd8fdfc
This change ensure that the compatibility test work again.
changelog_begin
changelog_end
* Correctly document and report malformed party names when allocating
changelog_begin
[Ledger API] Documented the hard-coded limit of 255 characters for Daml-LF party names.
[Ledger API] Malformed party names are now correctly reported back with an INVALID_ARGUMENT error
changelog_end
* Make sure that what is given is a suggestion
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* Use Future.successful where possible
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* Address https://github.com/digital-asset/daml/pull/8642#discussion_r565116571
* Address https://github.com/digital-asset/daml/pull/8642#discussion_r565122362
* Keep fields private where possible
* Whitelist new tests on compatibility suite
* Don't run the new tests on Canton
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
* Replace many occurrences of DAML with Daml
* Update docs logo
* A few more CLI occurrences
CHANGELOG_BEGIN
- Change DAML capitalization and docs logo
CHANGELOG_END
* Fix some over-eager replacements
* A few mor occurrences in md files
* Address comments in *.proto files
* Change case in comments and strings in .ts files
* Revert changes to frozen proto files
* Also revert LF 1.11
* Update get-daml.sh
* Update windows installer
* Include .py files
* Include comments in .daml files
* More instances in the assistant CLI
* some more help texts
This commit fixes a few copyright headers that have been missed in the
automatic update on Jan 1, as well as the generation code in the compat
workspace so it generates the right headers.
CHANGELOG_BEGIN
CHANGELOG_END
follow-up to #8099. If we don’t load the 1.dev dar in sandbox which is
totally sensible, we obviously also cannot run tests against it. In
newer test tool versions that’s not an issue but for older versions we
still need to exclude that one test.
changelog_begin
changelog_end
By definition, dev doesn’t guarantee any kind of compatibility so
trying to load it into Sandbox can fail. somewhat surprisingly it only
started failing now which I guess shows that we haven’t changed LF too
much recently.
changelog_begin
changelog_end
* Participant pruning ledger api server support ported from Canton
CHANGELOG_BEGIN
- [Ledger API]: The preview of `ParticipantPruningService` enables ledger participants to prune the "front" of ledger state at the participant including the ledger api server index.
CHANGELOG_END
* Review feedback from Stefano
* Add pruning tests plus missed command completions change
* Review feedback from Robert
* Improved test readability by having populate helper return offsets
* Review feedback
* Ledger api changes to pruning api and disable canton pruning test
- Change return result to PruneResponse
- Change type of PruneRequest.prune_up_to to string
* Review feedback: Use ApiOffsetSConverter for logged offsets
