This PR makes possible to check for contract IDs suffix during
preprocessing.
This is the first part of the task 3 described in #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* Generate short to long name mapping in aspect
Maps shortened test names in da_scala_test_suite on Windows to their
long name on Linux and MacOS.
Names are shortened on Windows to avoid exceeding MAX_PATH.
* Script to generate scala test name mapping
* Generate scala-test-suite-name-map.json on Windows
changelog_begin
changelog_end
* Generate UTF-8 with Unix line endings
Otherwise the file will be formatted using UTF-16 with CRLF line
endings, which confuses `jq` on Linux.
* Apply Scala test name remapping before ES upload
* Pipe bazel output into intermediate file
Bazel writes the output of --experimental_show_artifacts to stderr
instead of stdout. In Powershell this means that these outputs are not
plain strings, but instead error objects. Simply redirecting these to
stdout and piping them into further processing will lead to
indeterministically missing items or indeterministically introduced
additional newlines which may break paths.
To work around this we extract the error message from error objects,
introduce appropriate newlines, and write the output to a temporary file
before further processing.
This solution is taken and adapted from
https://stackoverflow.com/a/48671797/841562
* Add copyright header
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Augment completion.proto with deduplication-related info
CHANGELOG_BEGIN
CHANGELOG_END
* Explicitly specify fields not yet filled in when building Completion
* Time-based deduplication periods are measured in record time of completions
* Add deduplication_offset as a deduplication_period option
* Don't skip proto field numbers
* CompletionFromTransaction: use default Completion constructor
* submission_rank: reserve proto field for future use
* Add comment about reserved proto field
Previously, if the max deduplication time was extended, the participant
_might_ retain the old time for certain submissions.
CHANGELOG_BEGIN
- [Ledger API Server] The API server manages a single command tracker
per (application ID × submitters) pair. This tracker would read the
current ledger configuration's maximum deduplication time on creation,
but never updated it, leading to trackers that might inadvertently
reject a submission when it should have been accepted. The tracker now
reads the latest ledger configuration.
CHANGELOG_END
* Time conversion duration between buffer event and API domain transaction
* Compute partiesTemplates inversion mapping outside events transformation
* Other small optimizations
CHANGELOG_BEGIN
CHANGELOG_END
* Addition of a key_hash field to speed up fetchByKey queries
CHANGELOG_BEGIN
CHANGELOG_END
* changes to make key_hash and Optional field
CHANGELOG_BEGIN
- Update schema version for http-json-api query store with new key_hash field
- Improved performance for fetchByKey query which now uses key_hash field
CHANGELOG_END
* remove btree index for postgres and other changes based on code review comments
This PR enforces that LF Comparisons (both Ordering and Equality)
fails at runtime if comparing local vs global CID.
This corresponds to the second task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
The current README dates back to when `create-daml-app` was its own
repo, and seems mostly written from the perspective of being the default
home page for the project. These days, `create-daml-app` serves two
purposes:
1. As the core starting point for the Getting Started Guide, we can
assume people who are totally new to Daml will be following the
getting started guide. How else would they even know about this
template? In that case, I think it's fair to assume the user will *not*
be looking to the README for guidance. Therefore, repeating the
documentation here seems a bit pointless.
2. As a template to quickstart the creation of a "custom" app that is
not the GSG. For that use-case, the README is the place where people
would look for information, but at that point the information they're
looking for are more along the lines of quick refreshers of what
commands to run than explanations on how to use the app as Alice and
Bob.
CHANGELOG_BEGIN
CHANGELOG_END
* Add StorageBackend tests
changelog_begin
changelog_end
* Fix Oracle tests
* Do not use empty byte arrays
* Format
* Fix after rebase
* Substitute type params with type bounded ingest method
* Remove empty line
* Assert on configuration contents
* Fix Oracle build
* Add tests for ingestion initialization
* fmt
* Add test for leftover data after reset
* Add resetAll
* Use resetAll between tests
Co-authored-by: Marton Nagy <marton.nagy@digitalasset.com>
* participant-integration-api: Add logging to RecoveringIndexerSpec.
It's flaky, and I would like to know exactly what's going on here.
* participant-integration-api: Attempt to fix RecoveringIndexerSpec.
The checks for the health status are racy. I'm hoping increasing the
timeouts will help a little.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Inline timeouts in RecoveringIndexerSpec.
The `LedgerConfigurationSubscriptionFromIndexSpec` was flaky due to
stubs not specifying all behavior for `IndexConfigManagementService`.
This fixes the underlying issue by avoiding stubs in favor of fakes,
which means that _all_ behavior must be modelled.
Note: Martin Fowler has an excellent, terse description of [the various
forms of test doubles][TestDouble].
[TestDouble]: https://www.martinfowler.com/bliki/TestDouble.html
CHANGELOG_BEGIN
CHANGELOG_END
* Update the channels link in Upgrading.md
* Update the nixpkgs-unstable commit
CHANGELOG_BEGIN
CHANGELOG_END
* Fix new `hlint` warnings.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Reminder to put and empty line between subject and body
changelog_begin
changelog_end
* Update ci/check-changelog.sh
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Upgrade to a newer canton version (post 0.27.0 snapshot version)
with canton-community configuration that supports higher throughput.
changelog_begin
changelog_end
* Disable flaky reject DeeplyNestedValueIT:Reject tests that time out half the time
As stated in #10504 the contract ID freshness check cannot be
implemented correctly in general.
This PR drops the support for this (buggy) check.
This corresponds to the fist task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
This was limited a while back during the initial development of
_ledger-on-sql_ for reasons I can't remember. Let's stop doing that.
CHANGELOG_BEGIN
CHANGELOG_END
* Refactor ParameterStorageBackend
- a single method for atomic ledger initialization
- a single method to look up the ledger end
changelog_begin
changelog_end
* Add a test
* Fix reading event sequential ids
* Remove debug statements
* Allow ledgerEnd on an empty database
* Initialization is not safe to call concurrently
* Remove leftovers from isolation level change
* Use unit return type
for initialization methods
* Allow getParticipantId on an empty database
* Use exceptions instead of a return type ADT
* Don't use Try for initialization
* Clean up parameters table
* Simplify parameter storage api
* Address review suggestion
* Address review comment
* Address review comment
* Prefer ledger id over participant id mismatch
* Address review comment
* Move type definition
* Remove useleess new keyword
* Renove unused import
* Inline result mapping
* Fix reporting of mismatching participantId
* participant-integration-api: Construct completions in one place.
* sandbox-classic: Inline `CompletionFromTransaction#apply`.
It's only used here; there's no reason to keep it in the
_participant-integration-api_.
* participant-integration-api: Store a status gRPC protobuf.
Instead of storing the status code and message, we store a serialized
`google.rpc.Status` protocol buffers message. This allows us to pass
through any additional information reported by the driver `ReadService`.
The migration is only done for the append-only database, and preserves
old data in the existing columns. New data will only be written to the
new column.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Improve comments in migrations.
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
* participant-integration-api: Further improvements to migrations.
* participant-integration-api: Store the rejection status as 3 columns.
Serializing the details but keeping the code and message columns
populated.
* participant-integration-api: Publish the indexer protobuf to Maven.
Co-authored-by: Fabio Tudone <fabio.tudone@digitalasset.com>
* Use `extra` in the port file runner, rather than `temporary`.
* ledger-api-test-tool-on-canton: Use the port check runner.
Much simpler than the port file runner for our purposes.
* Replace `runner` with `runner_with_port_file`.
Rather than expecting a particular set of command-line-arguments, we use
templating.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename the `runner_with_port_check` target to the default.
* Use the port file and dynamic port generation in client/server tests.
This creates a runner named `runner_with_port_file` which knows how to
interpolate two variables, `%PORT_FILE%` and `%PORT%`. This allows us to
use the `port-file` argument to the kvutils runner rather than
hard-coding a port for conformance tests.
For now, we only use this for generating the kvutils reference ledger
export.
CHANGELOG_BEGIN
CHANGELOG_END
* Simplify the runner_with_port_file considerably.
It doesn't need to check if the port is open; we trust that the process
will do it.
This also makes sure the port file will be cleaned up, and reduces the
number of dependencies by making use of more functions in `extra`.
* Simplify port file generation in the new client-server runner.
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Simplify the runner_with_port_file further.
This doesn't need to work if the server doesn't take a port file.
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
It currently has all it needs for the docs to build, but not quite for
people to use it as a useful result of the triggers docs page: without
the UI components, it's hard to see anything the triggers are supposed
to do.
CHANGELOG_BEGIN
CHANGELOG_END
@nickchapman-da is taking care of 1.17.0-snapshot.20210817.7604.0.0c187853 (#10605), so they get pushed back to the end of the line.
Please do not merge this before #10605.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Move `DeduplicationPeriod` to ledger-api-domain so that it ca be reused and passed down from the ledger-api-client to into the v2.SubmitterInfo
CHANGELOG_BEGIN
participant-state - move `DeduplicationPeriod` to ledger-api-domain
CHANGELOG_END
* Revert unrelated changes
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
Commit log:
```
0c1878530b test: test for platform independent dars (#10535)
3822a8c886 ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de564f Introduce dependency on Oracle image version (#10597)
f8c0a35940 rewrite trigger docs to follow gsg (#10509)
aec601f929 Fix badly versioned empty tx (#10596)
d92440471c daml ledger export: export all parties (#10588)
fb09b72f96 Improve divulgence warning message. (#10595)
fd9d8724fc Upgrade to Oracle 19.12 (#10589)
386965c466 Upgrade Flyway to v7. (#10594)
654d2eee1b Bump url-parse to address dependabot alerts (#10593)
01b8e3011f Simplify loading of logback file (#10592)
19bfdbee00 update NOTICES file (#10591)
5f7a3691a5 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f32d4 Upgrade path-parse to 1.0.7 (#10587)
5c8025269d ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccfc4e Normalize transactions & values as a separate pass (#10524)
99e1d78bb8 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc39cb Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2b96 Skip subject in changelog check (#10584)
244262dfac Handle imports for builtin types correctly in Daml export (#10575)
1fc281f07e ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15465 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb933 JSON API: log ledger connection errors at every attempt (#10581)
933b58e3aa Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef894447f Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52cdc changelog_begin (#10579)
dc4629fff9 update NOTICES file (#10580)
8b0a0e7bbc update NOTICES file (#10578)
4b8b67a1b5 Upgrade Scalatest to v3.2.9. (#10576)
41e60f7302 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898cc9 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90dffd Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc6d6 rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44fbd update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa7c0 Bump bazel max jvm memory (#10569)
1cc136c033 update NOTICES file (#10565)
c69880ccf1 ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f80c Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f36b6 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* ledger-api-test-tool: Make IntelliJ happy with TransactionServiceIT.
* ledger-api-test-tool: Collect TransactionServiceIT tests into groups.
* ledger-api-test-tool: Split TransactionServiceIT into lots of suites.
CHANGELOG_BEGIN
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
been split into many test suites. If you are including or excluding
it, you will need to use the new test suite names, or you can use
"TransactionService" as a prefix for all of them.
If you are including or excluding individual tests, you will need to
update your arguments with the new test suite. You can find the new
test suite by running the test tool with the ``--list-all``
flag and looking for the test's short identifier. The short
identifiers have not changed, with the exception of
``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
is now in the ``ContractKeysIT`` test suite.
CHANGELOG_END
* rewrite trigger docs to follow gsg
Per #10419 point 4, I've rewritten the Triggers section to build upon
the Getting Started Guide instead of inventing its own example.
Compared to #10395, this has a lot more explanations as this page must
now serve the dual purpose of being a possible "next step" from the GSG
and being the main reference page for triggers. It's also lost the "next
steps" section, which I think is a bit of a shame, but it doesn't really
make sense here.
There's also no easy way for people not interested in the GSG to follow
along; should we expose the "completed GSG" as a tempate?
CHANGELOG_BEGIN
CHANGELOG_END
* keep copy-trigger as a template
* fix copy-trigger project name
* make up gsg-trigger template
* remove awkward sentence, fix existing typo
* update code to use when{,Some}
* add to
* swap emitCommands and getCommandsInFlight
* typo
* insist on state-correction perspective
* fix copy-trigger tests
* add back copy-trigger to whitelist
* add gsg-trigger to whitelist
* build correctly versioned empty tx
* dont call normalizeTx from Engine.validate
* dont recalc the tx-version in normalizeTx; just assume it is correct
CHANGELOG_BEGIN
CHANGELOG_END
* Add test-case to ConfigSpec for output type
changelog_begin
changelog_end
* Add an --all-parties flag to ledger export
changelog_begin
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
changelog_end
* Update docs
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
