* Make stable packages dependent on supported LF versions
Note that this just introduces an API dependency, we are not yet
filtering the list of packages. For that, I’d really like to
autogenerate StablePackages.scala since I don’t want to make that
manually maintained list even more complex. But review seems easier if
we first change the usage sites and then switch to autogenerating than
trying to do both in one go.
changelog_begin
changelog_end
* Fix script export tests
changelog_begin
changelog_end
* Allocate parties sequentially in script export tests
We’ve seen a few timeouts so this seems at least worth a try.
changelog_begin
changelog_end
* Extend logging to ease debugging
changelog_begin
changelog_end
* Remove the mutating schema
changelog_begin
- [Participant] All participants now use the new append-only schema. Existing databases will
automatically upgrade to the new schema the first time a participant/ledger is started.
changelog_end
* Fix post-commit validation test
* Remove append-only flags from CI
* Don't crash when using deprecated flag
* Increase timeout for reset service tests
* Fix typos in parameter names
* Restore removed test
* Restore removed CLI check
* Improve CLI parameter description
Add support for `--max-inbound-message-size` flag to the [Ledger Export](https://docs.daml.com/tools/export/index.html) tool.
CHANGELOG_BEGIN
- [Daml Assistant] Add support for `--max-inbound-message-size` flag to the Ledger Export tool.
CHANGELOG_END
- Add support for specifying either 1.2 or 1.3 as minimum TLS versions for ledger api server.
- Log enabled protocols (~TLS versions) and cipher suites at server and client startup.
- Add integration tests against Sandbox-classic and Sandbox
CHANGELOG_BEGIN
Sandbox: Add CLI flag to select minimum enabled TLS version for ledger API server.
CHANGELOG_END
Since we switch to scala 2.13, ImmArray companion object extends
`Factory`. Hence:
- the `apply` methods of `ImmArray` override the one from `Factory`
- we can use the notation `.to(ImmArray)` to convert an `Iterable` to
`ImmArray`
This PR drops those `apply` ImmArray. Conversion from Iterable to
`ImmArray` should use the `.to(ImmArray)`.
CHANGELOG_BEGIN
CHANGELOG_END
* Use `extra` in the port file runner, rather than `temporary`.
* ledger-api-test-tool-on-canton: Use the port check runner.
Much simpler than the port file runner for our purposes.
* Replace `runner` with `runner_with_port_file`.
Rather than expecting a particular set of command-line-arguments, we use
templating.
CHANGELOG_BEGIN
CHANGELOG_END
* Rename the `runner_with_port_check` target to the default.
* Add test-case to ConfigSpec for output type
changelog_begin
changelog_end
* Add an --all-parties flag to ledger export
changelog_begin
* [Daml export] You can now set the ``--all-parties`` option to generate
a ledger export as seen by all known parties.
changelog_end
* Update docs
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* LF 1.6 ledger export integration test
Test that Daml ledger export script handles DALF packages in LF version
1.6. These packages don't contain metadata, meanint `--package` flags
have to use the hash rather than the package name, and they contain no
type class instances, meaning the export script needs to contain newly
defined instances for type classes required to issue ledger commands in
Daml script.
changelog_begin
changelog_end
* Expose unconstrained Daml script commands
This will be used in Daml ledger exports of contracts defined in Daml LF
versions before 1.8. These versions did not include typeclass instances,
meaning that instances such as HasTemplateTypeRep need to be recreated
in the export Script. Not all instances required in the `Template t`
constraint can be recreated at the use-site. E.g. `HasSignatory`. By
avoiding these kinds of constraints it is possible to handle these cases
in Daml ledger exports anyway.
changelog_begin
changelog_end
* Identify all templates with missing instances
changelog_begin
changelog_end
* Use internal*Cmd on templates missing instances
changelog_begin
- [Daml export] Daml ledger export now handles templates in packages
using LF versions 1.7 or older. These package versions don't include
type class instances and Daml ledger export needs to generate
replacement instances in the generated script. The generated script
uses less type-safe versions of Daml script ledger commands.
changelog_end
* Add encodeType to encode Ast.Type in ledger export
This will be required to encode HasContractKey instances.
changelog_begin
changelog_end
* Simplify newline handling in encodeExport
* Encode missing template type class instances
* ZIP entries must use forward slash as path separator
* Throw error on encode of Any
Addressing
https://github.com/digital-asset/daml/pull/10526#discussion_r685807454
* Qualify choice using argument type
Addresses review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685817745
* Use lf.language.Util
Addresses
https://github.com/digital-asset/daml/pull/10526#discussion_r685827150
* Factor out header comment in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685830832
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
changelog_begin
changelog_end
This test suite performs the same test with different parameters three
times. These tests each setup some state on the ledger, then create a
Daml ledger export script, compile it, run it, and compare the new
ledger state to the previous state.
Running all this three times in the same target get exceed the default
timeout of 300s in some case. This splits these tests into three
separate targets to reduce the risk of timeouts.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Add regression test for exerciseByKey after exercise
changelog_begin
changelog_end
The discovery of exerciseByKey commands currently doesn't handle the
case where the contract or contract key is modified during the same
transaction. Issuing a regular exercise command in this case is an error
as it is not possible to reference a contract id that is generated
during the same transaction.
* Track contract-keys from exercise events
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Expose set of stable-packages package-ids
Needed in Daml ledger export to decide whether to depend on a package
without metadata or not.
changelog_begin
changelog_end
* Handle package dependencies without metadata in Daml export
This addresses the immediate issue reported in
https://github.com/digital-asset/daml/issues/10435.
DALF before LF version 1.8 don't contain metadata such as the package
name and version. However, Daml ledger export assumed that metadata was
available to generate `--package` flags for data-dependencies.
This change generates `--package=<hash>` flags for such dependencies on
packages that don't have metadata available.
This change requires additional care when checking if a package is a
stable-package. As the package name is not available without metadata we
cannot just check against names such as `daml-stdlib`. Instead this
change uses the list of stable-package package-ids exposed by
`com.daml.lf.language.StablePackages.Ids`, which was introduced in
0da814d250 (diff-208c6948c7c7f2b582faa8e4792bfe78b231188881947ad0cdd5046e0b9a40e7)
Note that this does not fully resolve#10435. Building a generated Daml
ledger export script will fail on missing instances of standard classes
such as `HasTemplate` because packages prior to LF version 1.8 don't
contain type-class instances. This will be addressed in a separate PR.
changelog_begin
changelog_end
* Fix Scala 2.12
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
That is the earliest LF version at which Daml Script is available,
according to 50c7b79f83/daml-lf/language/daml-lf.bzl (L71).
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Previously data-dependencies in the generated daml.yaml file had a path
including the output-dir path itself. E.g.
```
data-dependencies:
- /some/out/path/deps/some.dalf
```
This worked fine so long as the output path was absolute. However, if
the output path is a relative path, then it might lead to "no such file"
errors during daml build.
This change always makes the generated data-dependencies paths relative
to the daml.yaml file.
changelog_begin
- [Daml export] The generated paths to data-dependencies DALFs are now
relative to the generated daml.yaml. Fixes
https://github.com/digital-asset/daml/issues/10378.
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
* Pretty print daml.yaml in Daml ledger export
changelog_begin
changelog_end
* Use dynamic port in example-export generation
* Add daml.yaml to Daml ledger export golden test
* Sort data-dependencies for reproducibility
* Replace hashes in data-dependencies with a placeholder
To avoid test failure when any of these change.
* normalize data-dependencies path in scala client
So that the bazel build action is more reproducible itself.
* Fix Scala 2.12 build
* Factor out deleteRecursively
changelog_begin
changelog_end
* Generate ledger export in temp-dir
Build actions are not sandboxed in Windows. The ledger export will also
created a directory `deps` to store all the DALFs in. On Windows this
directory may persist across Bazel builds and may cause errors due to
attempts to overwrite an existing directory.
To avoid these issues the ledger export is generated into a temporary
directory and only the wanted files are moved to the expected output
locations in the bazel execroot afterwards.
* Close source
* Use replace instead of replaceFirst
The latter matches regex pattern instead of just substring matching
which fails with Windows paths due to `\` being interpreted as an escape
character.
* Normalize Windows \ to Unix /
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* client_server_build - user defined outputs
Let the user define a list of output files on client_server_build.
changelog_begin
changelog_end
* daml-script runner expose main(config)
changelog_begin
changelog_end
* Add an example Daml ledger export to the docs
* Build Daml ledger export
changelog_begin
changelog_end
* sh_inline_test
Support toolchain arguments to sh_inline_test. Usefuly for make variable
expansion, e.g. for the POSIX toolchain.
* Add a golden test for Daml ledger export
* Test args files as well
* Use sed from POSIX toolchain
* Add normalization comment to top-level comment
* Ignore trailing CR on Windows
The JSON formatting of the args file uses Windows line endings on
Windows. Therefore, the args.json output technically differs from the
expected output. We're happy to ignore the difference in CRLF here.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* Add token parameter to daml export
changelog_begin
changelog_end
* Add test case for --access-token-flag
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Factor out all encoding components
* Provide TextMap party mapping instead of record
This allows users to customize the mapping from old party names to new
party names.
* Emit bindings for parties at beginning of export
For better readability, to avoid `getParty "Alice" parties` all over the
place.
* getParty --> lookupParty to avoid conflict with Prelude
* Use DA.Traversable.mapA over TextMap
* Update integration test
* Update args.json
* Update the daml export docs
changelog_begin
* [Daml export] Users can now define a mapping from parties in the
original ledger state to parties to be used when recontructing the
ledger state. The ``parties`` component of the argument to the
generated export script now takes a mapping from old party name to new
party name.
changelog_end
* getContract --> lookupContract for consistency with lookupParty
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Add test suite for CLI parser and fix --end
changelog_begin
changelog_end
* Accept a comma separated list on --party
changelog_begin
changelog_end
* Use scalaopts list parsing support
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Take user defined mappings for unknown contracts
* Remove unused `allocateParties` helper
These are now provided in `args.json`.
* No longer fail on missing contract ids
* Generate a default args.json
* Configure daml start to run the export
Configures init-script and script-options such that `daml start` will
execute the export script with the default `args.json` input.
Configures navigator to display the default parties.
* Update daml export integration test
Input format has changed.
* Update documentation for daml ledger export
changelog_begin
* [Daml export] Daml ledger export now handles references to missing
contracts. The generated export script takes a mapping from unknown
contract ids to replacement contract ids as a parameter. Users can
prepopulate the ledger with replacements of missing contracts and
configure the export script to reference these replacements.
The generated export script no longer exposes ``testExport`` as an
entry point, instead Daml export generates a default input file
``args.json`` that can be used to execute the ``export`` function from
``daml script`` or ``daml start``.
changelog_end
* Reinstante testExport & allocateParties
Convenient for testing in Daml IDE or on ledgers that don't have
automatic party allocation.
* Add docstrings to the generated Daml export script
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Merge ACS encoding with transactions
Both can be first converted to Actions and only then encoded.
* Check for unknown contract id references in ACS and tree
* Separate construction and encoding of export
* Factor out export actions encoding
* inline encodeTransactionTreeStream
* Add test-cases for cid references in export
* Don't crash on unknown contract reference in ACS
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* Add setTime flag
changelog_begin
* [Daml export] Enable the ``--set-time`` flag on
``daml ledger export script`` to replicate transaction time stamps.
This only works on ledgers operating in static time mode.
changelog_end
* Dump.scala --> Export.scala
So that the file name and class name match.
This was forgotten in 5e652bb2f8.
* Implement optional setTime actions
* Define Action type encompassing SetTime and Submit
Factor out construction of submit actions interleaved with setTime
actions.
This makes it easier to test the corresponding functionality in
isolation.
* EncodeSetTimeSpec
* ActionsFromTreesSpec
* update docs on daml ledger export
* Enable --set-time in integration test
To ensure that the generated code actually compiles
* Fix date/time encoding
Requires additional imports for DA.Time and DA.Date.
Requires qualified references to DA.Time.time, DA.Date.date, and
DA.Date.MONTH.
* Fix Scala 2.13
* Move setTime parameter to Action.fromTrees
* Move export transaction time docs out of caveats
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* rename "dump" to "export"
* Add Daml export command to assistant
* Make daml export script a subcommand of daml export
* daml-assistant IT for daml export script
* Expose export as daml ledger command (hidden for now)
Add Haskell side parser for ledger export flags
changelog_begin
* [Daml export] New feature: Use ``daml ledger export script`` to
generate a Daml script that will reconstruct a given ledger state.
This is an early access feature.
changelog_end
* Update integration test
* Remove daml export - it's daml ledger export now
* integration-test set ledger port
The integration tests don't configure the port in the project config,
but on the command-line, and they don't use the default value.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
