* interfaces: scala typechecker implementation
This is the scala side of the lf typechecker for interfaces.
CHANGELOG_BEGIN
CHANGELOG_END
* added collision check
* added exercise/fetch typechecking
* review suggestions
* added todos for collision/typing scala tests
* Extracted common error implementations
* ErrorGroups
* TransactionError hierarchy
* LedgerApiErrors
* PackageServiceError
* ProtoDeserializationError
* PruningServiceError
* SubmissionErrors
CHANGELOG_BEGIN
CHANGELOG_END
* Adapt extracted error groups and definitions to adhere to local tech stack:
* Use //ledger/error:error core API
* Use DAML SDK logging stack
* Manual rebase to latest changes from Canton
* Extracted RejectionGenerator from Canton
* Adapted RejectionGenerator
* Added //ledger/error:error to artifacts
* Pass correlationId to errors
* Pass parameters as implicits to shave some lines and improve readability
* Workaround for encountered Scala/JDK8 bug
* Addressed Ratko's review comments
* Separate exercise & fetch for interfaces from templates
part of #10810
changelog_begin
changelog_end
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
* Update compiler/daml-lf-ast/src/DA/Daml/LF/Ast/Base.hs
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
Co-authored-by: Sofia Faro <sofia.faro@digitalasset.com>
99% of our usecases use Value[ContractId] so this PR just fixes it.
The few other usescases are:
1. Value[Nothing] which we use for keys. This is technically more
precise but we benefit very little from it.
2. Value[String] mostly because in a few places we are lazy.
We don’t have any code which benefits from being polymorphic in the
contract id type.
changelog_begin
changelog_end
Following #10763, we drop the ad-hoc builders for `FrontStack`.
* Building a `Fronstack` from individuals elements should be done with
standard scala buidler.
* Building a `Fronstack` from a `TraversableOne` should be done with
the scala 2.13 `.to(FrontStack)` methd
* Building a `Fronstack` from a `ImmArray` should be done with the
`toImmArray` method.
CHANGELOG_BEGIN
CHANGELOG_END
* interfaces: protobuf encoder haskell side
This is the implementation of the protobuf encoder on the haskell side.
CHANGELOG_BEGIN
CHANGELOG_END
* missing decoder cases for added interface data constructor
* Enhance InMemoryLedger to use the ValueEnricher.
Allowing SandboxServer to run Engine with the default transactionNormalization=false
CHANGELOG_BEGIN
CHANGELOG_END
* enrich only when responding to verbose API queries
Since we switch to scala 2.13, ImmArray companion object extends
`Factory`. Hence:
- the `apply` methods of `ImmArray` override the one from `Factory`
- we can use the notation `.to(ImmArray)` to convert an `Iterable` to
`ImmArray`
This PR drops those `apply` ImmArray. Conversion from Iterable to
`ImmArray` should use the `.to(ImmArray)`.
CHANGELOG_BEGIN
CHANGELOG_END
This gets us into a state where PartialTransaction always has the
authorization context which is much more sensible than having the
committers in speedy while the rest is in ptx.
The reason for having the split in the first place was the scenario
service but now that we create a new speedy machine per submission
that’s no longer a valid reason.
changelog_begin
changelog_end
Create normalized TXs when a partial TX is finalised.
Except in limited cases! (i.e for scenario-runner, sandbox)
CHANGELOG_BEGIN
CHANGELOG_END
normalize values in the engine as they are converted from speedy-values
fix 2.12 build
backout redundant change
ensure byKey field is correctly normalized when constructed by engine
rename flag: valueNormalization -> transactionNormalization
improve comment
delete commented-out code
rename: toValueNorm --> toNormalizedValue
rename: (SValue.) toValue --> toUnNormalizedValue
revert changes to ptx so that the interface to insertCreate() etc is Value-based (not SValue-based)
improve comments
respell: toUnNormalizedValue --> toUnnormalizedValue
fix build
This PR makes possible to check for contract IDs suffix during
preprocessing.
This is the first part of the task 3 described in #10504.
CHANGELOG_BEGIN
CHANGELOG_END
This PR enforces that LF Comparisons (both Ordering and Equality)
fails at runtime if comparing local vs global CID.
This corresponds to the second task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade to a newer canton version (post 0.27.0 snapshot version)
with canton-community configuration that supports higher throughput.
changelog_begin
changelog_end
* Disable flaky reject DeeplyNestedValueIT:Reject tests that time out half the time
As stated in #10504 the contract ID freshness check cannot be
implemented correctly in general.
This PR drops the support for this (buggy) check.
This corresponds to the fist task of #10504.
CHANGELOG_BEGIN
CHANGELOG_END
* build correctly versioned empty tx
* dont call normalizeTx from Engine.validate
* dont recalc the tx-version in normalizeTx; just assume it is correct
CHANGELOG_BEGIN
CHANGELOG_END
* Normalize transactions & values as a separate pass. Use for simpler defintiion of isReplayedBy.
CHANGELOG_BEGIN
CHANGELOG_END
normalize transaction version
* remove stray import from bad merge which breaks scala 2_12 build
* change isReplayedBy to only norm its RIGHT (replay) argument
* add forgotton normalization for ValueEmum
* switch to use existing value normalization code (remove my newly coded duplicate code)
* normalize submittedTransaction before calling engine.validate
* dont call normalizeTx from Engine.validate
* *do* call normalizeTx from Engine.validate
* Upgrade Scalatest to v3.2.9.
Because of some coupling we also have to upgrade Scalaz to the latest
v7.2 point release, v7.2.33.
The Scalatest changes are quite involved because the JAR has been broken
up into several smaller JARs. Because Bazel expects us to specify all
dependencies and doesn't allow transitive dependencies to be used
directly, this means that we need to specify the explicit Scalatest
components that we use.
As you can imagine, this results in quite a big set of changes. They
are, however, constrained to dependency management; all the code remains
the same.
CHANGELOG_BEGIN
CHANGELOG_END
* http-json-oracle: Fix a Scalatest dependency.
* ledger-api-client: Fix a Scalatest dependency.
* LF 1.6 ledger export integration test
Test that Daml ledger export script handles DALF packages in LF version
1.6. These packages don't contain metadata, meanint `--package` flags
have to use the hash rather than the package name, and they contain no
type class instances, meaning the export script needs to contain newly
defined instances for type classes required to issue ledger commands in
Daml script.
changelog_begin
changelog_end
* Expose unconstrained Daml script commands
This will be used in Daml ledger exports of contracts defined in Daml LF
versions before 1.8. These versions did not include typeclass instances,
meaning that instances such as HasTemplateTypeRep need to be recreated
in the export Script. Not all instances required in the `Template t`
constraint can be recreated at the use-site. E.g. `HasSignatory`. By
avoiding these kinds of constraints it is possible to handle these cases
in Daml ledger exports anyway.
changelog_begin
changelog_end
* Identify all templates with missing instances
changelog_begin
changelog_end
* Use internal*Cmd on templates missing instances
changelog_begin
- [Daml export] Daml ledger export now handles templates in packages
using LF versions 1.7 or older. These package versions don't include
type class instances and Daml ledger export needs to generate
replacement instances in the generated script. The generated script
uses less type-safe versions of Daml script ledger commands.
changelog_end
* Add encodeType to encode Ast.Type in ledger export
This will be required to encode HasContractKey instances.
changelog_begin
changelog_end
* Simplify newline handling in encodeExport
* Encode missing template type class instances
* ZIP entries must use forward slash as path separator
* Throw error on encode of Any
Addressing
https://github.com/digital-asset/daml/pull/10526#discussion_r685807454
* Qualify choice using argument type
Addresses review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685817745
* Use lf.language.Util
Addresses
https://github.com/digital-asset/daml/pull/10526#discussion_r685827150
* Factor out header comment in tests
Addressing review comment
https://github.com/digital-asset/daml/pull/10526#discussion_r685830832
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Expose set of stable-packages package-ids
Needed in Daml ledger export to decide whether to depend on a package
without metadata or not.
changelog_begin
changelog_end
* Handle package dependencies without metadata in Daml export
This addresses the immediate issue reported in
https://github.com/digital-asset/daml/issues/10435.
DALF before LF version 1.8 don't contain metadata such as the package
name and version. However, Daml ledger export assumed that metadata was
available to generate `--package` flags for data-dependencies.
This change generates `--package=<hash>` flags for such dependencies on
packages that don't have metadata available.
This change requires additional care when checking if a package is a
stable-package. As the package name is not available without metadata we
cannot just check against names such as `daml-stdlib`. Instead this
change uses the list of stable-package package-ids exposed by
`com.daml.lf.language.StablePackages.Ids`, which was introduced in
0da814d250 (diff-208c6948c7c7f2b582faa8e4792bfe78b231188881947ad0cdd5046e0b9a40e7)
Note that this does not fully resolve#10435. Building a generated Daml
ledger export script will fail on missing instances of standard classes
such as `HasTemplate` because packages prior to LF version 1.8 don't
contain type-class instances. This will be addressed in a separate PR.
changelog_begin
changelog_end
* Fix Scala 2.12
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
part of #9974
- rename `msg` field to `message`
- drop `engine.Error.Validation.Generic`
- rename argument `where` to `location`
- Builtin set the `dependsOnTime` before throwing the callback `SResultNeedTime`
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `LoggingEntries` a non-case class.
There's no reason for it to need `equals`, etc.
CHANGELOG_BEGIN
CHANGELOG_END
* ledger-api-domain: Convert commands into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This also adds a couple of missing items, and always logs `workflowId`.
* participant-state: Convert updates into a logging value.
Instead of having a function, let's use `ToLoggingContext`.
This changes some of the logging context structure, but otherwise
everything remains the same.
* Make sure Scaladoc is lined up for modified code.
* Stable packages bypass version restrictions.
Part of #10029
changelog_begin
changelog_end
* Add missing bypass
* scalafmt
* Fix tests
* fix script runner
* Fix scenario service
* Dont make stable packages configurable
* unused import
* revert unnecessary formatting changes
* Added a StableOnly engine mode and a conformance test that uses it
* buildifier-fix
* Inline LanguageVersions.StableOnly
* scalafmt
* Remove EngineMode in favor of representing the version range directly
* Add -unsafe to flag name
* Apply suggestions from code review
We drop this check for the following reasons:
* Its only usage was inside the transaction builder, this is subsumed
by the check we added in the SValue to Value translation (#10370)
* It seems to origianl check more that nesting (and so to be
overcomplicate)
* It was used in non-consitent way. It was used to check create
arguments and choice arguments, but niether choice result nor
contract key.
CHANGELOG_BEGIN
CHANGELOG_END
The conversion of SValue to Value already ensures the resulting value
has a serializable type. Here we add a check to ensure it does not
overpass the maximum allow nesting.
* Limit length of package ids to 64 characters
In practice, you cannot use a package id that is not a valid sha256
outside of a test anyway since we validate them. However, we didn’t
enforce it at the daml-lf level which is a bit annoying.
This PR fixes this and adds a length restriction of 64
characters (i.e. 256 bytes for sha256).
We could go a bit further and also restrict the characters to valid
sha256 hashes (i.e. 64 character hex strings). I don’t feel all that
strongly about that either way. We use other characters in tests but
fixing that shouldn’t be particularly hard either. The extra
characters don’t seem to cause problem so far, so I think it’s fine to
keep that.
No changelog entry since I don’t see how a user can be affected by this.
changelog_begin
changelog_end
* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala
* participant-state: Remove the aliases to `Ref.LedgerString`.
Use the aliases directly in `Ref` instead.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-lf/data: Improve comments on ID aliases.
* daml-lf/data: Fix a typo in a comment.
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
Co-authored-by: Miklos <57664299+miklos-da@users.noreply.github.com>
* Pass commitLocation along in engine warning log.
changelog_begin
changelog_end
* Use commitLocation in diagnostics.
* scalafmt
* update divulgence test
* Fix commitLocation and expected diagnostic locations
* daml-lf/data: Move ID aliases to `Ref` from _ledger-api-common_.
This allows us to remove a lot of dependencies on _ledger-api-common_,
and use these aliases in other places where that module is not used.
CHANGELOG_BEGIN
CHANGELOG_END
* participant-integration-api: Remove an unused import.
* http-json-oracle: Remove `ledger-api-common` as a dependency.
* bindings-rxjava: Remove a now-unused dependency.
* Remove Optional[Location] field from transaction nodes.
xoptLocation
working but very messy!
cleanup PartialTransaction code
pass locationInfo in CompleteTransaction
dont pass locationInfo in SumittedTransaction
pass optLocation to CheckAuthorization
temp reinstate xoptLocation in ExerciseContextInfo
cleanup Conversions.scala
remove Optional[Location] field from transaction nodes.
CHANGELOG_BEGIN
CHANGELOG_END
* fix bad merge
* fix build after merge
* rename
* doc comment, and question
* LF: change type from Try to Either in archive module
This is the first part of restructuring errors in archive module.
This is part of #9974.
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* remove type alias
* apply stephen suggestion
* fix after rebase
* fix test
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Separate traces from warnings in engine.
I decided to separate the engine warnings from the tracelog after all,
because I think it will make testing and maintenance easier in the
long run.
Part of #9947, follow up from #10179
changelog_begin
changelog_end
* scalafmt
* Apply suggestions from code review
Co-authored-by: Remy <remy.haemmerle@daml.com>
* dont use case class for WarningLog
* revert TraceLog changes from last PR
* Scala 2.12 doesnt have ArrayBuffer.addOne :(
* remove isWarnEnabled check
Co-authored-by: Remy <remy.haemmerle@daml.com>
* LF: Make DarReader ZipEntries immulatble
CHANGELOG_BEGIN
CHANGELOG_END
* optim
* Revert "optim"
This reverts commit 3b25a8c68b.
* a function should not close a stream it didn’t open
* typo
* we do not support dar without manifest !
* LF: Simplify archive reader.
- decouple Reader and Decoder
- introduce case class to handle hash, proto payload, and version
CHANGELOG_BEGIN
CHANGELOG_END
* Address Moritz' review
* cosmetic
We probably want to start displaying the full ptx on the client side
but for now this at least moves things out of speedy and into the
rendering layer where they belong.
It also fixes IncompleteTransaction to unwind properly so the roots
are really the roots.
changelog_begin
changelog_end
* Use PartialFunction for more concise code.
CHANGELOG_BEGIN
CHANGELOG_END
* tweak to use guards
* blind fix for 2.12
* try fix 2.12 build
* loose if-guard parens
* fixes for 2.12 build (should actually work this time!)
* Add divulgence warning and test in script service.
Part of #9947, building on the key visibility checks from #10136
This PR adds a divulgence warning inside the traceLog.
I wasn't sure whether:
1. these warnings should be kept in a separate structure from the
traceLog, and therefore transmitted separately, or
2. these warnings should be kept together with traces, but the severity
should be tracked and also transmitted over grpc, and warnings should
be logged as warnings instead of as debug messages, or
3. these warnings should be kept together with traces, but
logged as warnings instead of debug messages,
4. these warnings should be treated exactly like traces
I'm leaning toward #2, but this PR implements #3.
This PR tests the warning via the script service tests.
changelog_begin
changelog_end
* scalafmt
* Address Moritz's review
* Rename traceLog.add to traceLog.addDebug
* fix test
* Add test using exercise
* add single transaction test
* Testing for Validation.isReplayedBy
changelog_begin
changelog_end
try fix scala 2_12 build
fetch node: samples and tweaks
list sig/insig field tweaks side-by-side so coverage is easier to eyeball
exerciseResult tweak sig & insig
add sig/insig tweak for byKey of Fetch node
add sig/insig tweak for key of Fetch node
must use ImmArray.empty instead of ImmArray() for scala 2.12
another blind attempt to fix scala 2.12
use for/yield instead of flatMap for list-comprehensions
test sig/insig tweaks to LookupByKey nodes
wont attempt to tweak rollbacks
(and shorten trailing dash in comment banners)
comment about how Tweak testing is organized
dont dep/import scalacheck (not used!)
no wild imports
* val --> private def
* def --> val
* prefer fail to println/assert(false)
* daml-lf/data: Truncate party names in log output, on request.
The party name can grow quite long, so we offer ledger implementors the
opportunity to truncate it in structured log output.
Unfortunately, because we use Logback through the global
`LoggerFactory`, there is no place to inject logging configuration. This
means we also need to use global, mutable state to configure logging
output. I have added a `LoggingConfiguration` class+object in Daml-LF
Data, which may not be the best place, but I can't think of a better
one right now. I suggest we leave it there until it has reason to grow,
at which point we may want to move it.
CHANGELOG_BEGIN
CHANGELOG_END
* logging-entries: Make `ToLoggingValue` mixin-able.
* participant-integration-api: Truncate parties in filters when logging.
* participant-integration-api: Cast to `Party` for logging.
Invalid input should not break the request at this point. No assertions.
* daml-lf/data: Move `Party to LoggingValue` to a new package.
This avoids the transitive dependency issue most of the time.
* daml-lf-data: Move the `Identifier` logging to another package.
Again, reduces the need for transitively depending on _logging-entries_.